Method, apparatus, and vehicle for operating a vehicle

Abstract

The present invention relates to a method, an apparatus, and a vehicle for operating a vehicle. A speed behavior planning for a vehicle, in particular, relates to a system for providing speed behavior planning for a vehicle with autonomous driving capabilities. The system includes identifying a plurality of speed limits along a trajectory; determining, by one or more processors, a plurality of associated candidate speed behaviors by connecting the speed limits along the trajectory; determining, by one or more processors, a determined speed behavior by using the plurality of candidate speed behaviors, wherein the determination includes smoothing at least one speed discontinuity between two connected speed limits along the trajectory; and operating, by one or more processors, a drive assembly of the vehicle according to the determined speed behavior.

Description

[0001] This application is a divisional application of the application with the application date of January 24, 2019, application number 201910069748.X, and invention title "Speed Behavior Planning for Vehicles". Technical Field

[0002] The present invention relates to speed behavior planning for vehicles. Summary of the Invention

[0003] Generally, in one aspect, a method includes: receiving current information related to speed behavior factors of a vehicle being driven; deriving candidate speed behaviors of the vehicle based on the speed behavior factors; and deriving a determined speed behavior based on the candidate speeds.

[0004] In some implementations of the method, receiving current information related to speed behavior factors may include: receiving signals from one or more sensors. Receiving current information related to speed behavior factors may include receiving information from a data source remotely, locally, or both. The speed behavior factors may include one or more of the following: legal regulations, moving objects, events, mechanical limitations of the vehicle, conditions of vehicle components, weather conditions, user preferences, obstructions, and road characteristics. In some implementations of the method, the candidate speed behaviors may include speed limits.

[0005] In some implementations of the method, the derivation of candidate speed behaviors or the derivation of determined speed behaviors may be applied along a trajectory. The derivation of candidate speed behaviors or the derivation of determined speed behaviors may be performed in a spatial domain, a temporal domain, or both.

[0006] In some implementations of the method, deriving candidate speed behaviors may include connecting speed limits along a trajectory. Deriving candidate speed behaviors may include identifying a minimum value of an aggregation of one or more given speed limits. Deriving candidate speed behaviors may include using a function to describe the speed behavior between the speed behavior factors and the vehicle. Deriving candidate speed behaviors may include using a function to smooth one or more speed discontinuities. In some implementations of the method, the function may associate speed with distance, time, or both. The function may include a linear function, a non-linear function, or both.

[0007] In some implementations of the method, smoothing one or more discontinuities may include smoothing a step-down discontinuity. Smoothing one or more discontinuities may include smoothing a step-up discontinuity. Smoothing one or more discontinuities may include ignoring the smoothing of a step-up discontinuity.

[0008] In some implementations of the method, deriving candidate speed behaviors or deriving determined speed behaviors may include using queuing processing to hold and arrange one or more candidate speed behaviors into a queue. The queue may be sorted based on the speed values of one or more speed behaviors. The queue may be time-dependent, space-dependent, or dependent on both.

[0009] In some implementations of the method, deriving determined speed behaviors may include using an optimization algorithm to find speed behaviors. Deriving determined speed behaviors may include optimizing the smoothness of acceleration. Deriving determined speed behaviors may include optimizing the smoothness of jerk. Deriving determined speed behaviors may include treating speed behaviors as hard constraints in the optimization algorithm. Deriving determined speed behaviors may include treating speed behaviors as soft constraints in the optimization algorithm. Deriving determined speed behaviors may include treating speed behaviors as partial soft constraints and partial hard constraints in the optimization algorithm. Deriving determined speed behaviors may include considering past driving speed behaviors. Deriving determined speed behaviors may include considering past determined speed behaviors. Deriving determined speed behaviors may include considering the current speed of the vehicle. Deriving determined speed behaviors may include maximizing the driving distance. Deriving determined speed behaviors may include minimizing the driving time. Deriving determined speed behaviors may include optimizing longitudinal speed behaviors. Deriving determined speed behaviors may include optimizing lateral speed behaviors. Deriving determined speed behaviors may include optimizing longitudinal speed behaviors and lateral speed behaviors together in the optimization process.

[0010] Implementations of the method may include communicating with one or more remote computing devices. The communication may include communicating with one or more mobile devices. The communication may include communicating with one or more user interface devices. The communication may include communicating with one or more remote operation servers. The communication may include communicating with one or more fleet management servers.

[0011] Implementations of the method may include driving the vehicle based on the determined speed behavior.

[0012] Implementations of the method may include iterating the activity at a time frequency of at least once per second. Implementations of the method may include iterating the activity at a spatial frequency of at least once per 1m. Implementations of the method may include iterating the activity at or before a bifurcation point in a given trajectory. Implementations of the method may include iterating the activity at or before a merge point in a given trajectory. Implementations of the method may include iterating the activity at or before an intersection point in a given trajectory.

[0013] Implementations of the method may include visualizing the determined speed behavior.

[0014] Generally, in one aspect, a vehicle includes: (a) a drive assembly including an acceleration assembly, a steering assembly, and a deceleration assembly; (b) an autonomous driving capability for sending signals to the drive assembly to drive the vehicle in at least a partially autonomous driving mode; (c) a planning assembly for receiving current information related to speed behavior factors, deriving candidate speed behaviors of the vehicle based on the speed behavior factors, and deriving a determined speed behavior based on the candidate speed behaviors; (d) a command assembly for causing the drive assembly to self-drive the vehicle based on the determined speed behavior.

[0015] In some implementations of the vehicle, receiving current information related to speed behavior factors may include receiving signals from one or more sensors. Receiving current information related to speed behavior factors may include receiving information from a data source remotely or locally or both. The speed behavior factors may include one or more of the following: legal regulations, moving objects, events, mechanical limitations of the vehicle, conditions of vehicle components, weather conditions, user preferences, obstructions, and road characteristics.

[0016] In some implementations of the vehicle, the candidate speed behaviors may include speed limits. The derivation of the candidate speed behaviors or the derivation of the determined speed behavior may be applied along a trajectory. The derivation of the candidate speed behaviors or the derivation of the determined speed behavior may be performed in a spatial domain or a temporal domain or both. Deriving the candidate speed behaviors may include connecting speed limits along a trajectory. Deriving the candidate speed behaviors may include identifying a minimum value of an aggregation of one or more given speed limits. Deriving the candidate speed behaviors may include using a function to describe the speed behavior between the speed behavior factors and the vehicle. Deriving the candidate speed behaviors may include using a function to smooth a discontinuity of one or more speeds. The function may relate speed to distance or time or both. The function may include a linear function or a non-linear function or both.

[0017] In some implementations of the vehicle, smoothing a discontinuity of one or more speeds may include smoothing a step-down discontinuity. Smoothing a discontinuity of one or more speeds may include smoothing a step-up discontinuity. Smoothing a discontinuity of one or more speeds may include ignoring the smoothing of a step-up discontinuity.

[0018] In some implementations of the vehicle, deriving the candidate speed behaviors or deriving the determined speed behavior may include using queuing processing to hold and arrange one or more candidate speed behaviors into a queue. The queue may be sorted based on the speed values of one or more speed behaviors. The queue may be time-dependent, space-dependent, or both.

[0019] In some implementations of the vehicle, deriving a determined speed behavior can include using an optimization algorithm to find the speed behavior. Deriving a determined speed behavior can include optimizing the smoothness of acceleration. Deriving a determined speed behavior can include optimizing the smoothness of jerk. Deriving a determined speed behavior can include treating the speed behavior as a hard constraint in the optimization algorithm. Deriving a determined speed behavior can include treating the speed behavior as a soft constraint in the optimization algorithm. Deriving a determined speed behavior can include treating the speed behavior as a partially soft and partially hard constraint in the optimization algorithm. Deriving a determined speed behavior includes considering past driving speed behaviors. Deriving a determined speed behavior can include considering past determined speed behaviors. Deriving a determined speed behavior can include considering the current speed of the vehicle. Deriving a determined speed behavior can include maximizing the driving distance. Deriving a determined speed behavior can include minimizing the driving time. Deriving a determined speed behavior can include optimizing the longitudinal speed behavior. Deriving a determined speed behavior can include optimizing the lateral speed behavior. Deriving a determined speed behavior can include co-optimizing the longitudinal speed behavior and the lateral speed behavior in the optimization process.

[0020] Implementations of the vehicle can include a communication component that communicates with one or more remote computing devices. The communication component can communicate with one or more mobile devices. The communication component can communicate with one or more user interface devices. The communication component can communicate with one or more remote operation servers. The communication component can communicate with one or more fleet management servers.

[0021] In some implementations of the vehicle, a planning component can iteratively process activities at a temporal frequency of at least once per second. The planning component can iteratively process activities at a spatial frequency of at least once per 1 m. The planning component can iteratively process activities at or before a bifurcation point in a given trajectory. The planning component can iteratively process activities at or before a merge point in a given trajectory. The planning component can iteratively process activities at or before an intersection point in a given trajectory.

[0022] Implementations of the vehicle can include a display for visualizing the determined speed behavior.

[0023] Generally, in one aspect, a device can include: a memory of instructions; and a processor operable according to the instructions to (1) receive current information related to speed behavior factors of a vehicle being driven, (2) derive candidate speed behaviors of the vehicle based on the speed behavior factors, and (3) derive a determined speed behavior based on the candidate speed behaviors.

[0024] In some implementations of the device, receiving current information related to speed behavior factors can include receiving signals from one or more sensors. Receiving current information related to speed behavior factors can include receiving information from a data source remotely, locally, or both. The speed behavior factors can include one or more of the following: legal regulations, moving objects, events, mechanical limitations of the vehicle, conditions of vehicle components, weather conditions, user preferences, obstructions, and road characteristics. Candidate speed behaviors can include speed limits.

[0025] In some implementations of the device, a derivation of a candidate speed behavior or a determination of a speed behavior derivation can be applied along a trajectory. The derivation of a candidate speed behavior or the determination of a speed behavior derivation can be performed in a spatial domain, a temporal domain, or both. Deriving a candidate speed behavior can include connecting speed limits along a trajectory. Deriving a candidate speed behavior can include identifying a minimum of an aggregation of one or more given speed limits. Deriving a candidate speed behavior can include using a function to describe the speed behavior between speed behavior factors and the vehicle. Deriving a candidate speed behavior can include using a function to smooth discontinuities of one or more speeds. The function can relate speed to distance, time, or both. The function can include a linear function, a non-linear function, or both.

[0026] In some implementations of the device, smoothing one or more discontinuities can include smoothing a discontinuity of a step-down. Smoothing one or more discontinuities can include smoothing a discontinuity of a step-up. Smoothing one or more discontinuities can include ignoring the smoothing of a discontinuity of a step-up.

[0027] In some implementations of the device, deriving candidate speed behaviors or deriving determined speed behaviors may include using queuing processing to hold and arrange one or more candidate speed behaviors as a queue. The queue may be sorted based on the speed values of one or more speed behaviors. The queue may be time-dependent, space-dependent, or both. Deriving determined speed behaviors may include using an optimization algorithm to find speed behaviors. Deriving determined speed behaviors may include optimizing the smoothness of acceleration. Deriving determined speed behaviors may include optimizing the smoothness of jerk. Deriving determined speed behaviors may include treating speed behaviors as hard constraints in the optimization algorithm. Deriving determined speed behaviors may include treating speed behaviors as soft constraints in the optimization algorithm. Deriving determined speed behaviors may include treating speed behaviors as partial soft constraints and partial hard constraints in the optimization algorithm. Deriving determined speed behaviors may include considering past driving speed behaviors. Deriving determined speed behaviors may include considering past determined speed behaviors. Deriving determined speed behaviors may include considering the current speed of the vehicle. Deriving determined speed behaviors may include maximizing the driving distance. Deriving determined speed behaviors may include minimizing the driving time. Deriving determined speed behaviors may include optimizing longitudinal speed behaviors. Deriving determined speed behaviors may include optimizing lateral speed behaviors. Deriving determined speed behaviors may include co-optimizing longitudinal speed behaviors and lateral speed behaviors in the optimization process.

[0028] Implementations of the device may include a communication component that communicates with one or more remote computing devices. The communication component may communicate with one or more mobile devices. The communication component may communicate with one or more user interface devices. The communication component may communicate with one or more remote operation servers. The communication component may communicate with one or more fleet management servers. The communication component may send commands to drive the vehicle based on determined speed behaviors.

[0029] In some implementations of the device, the processor may iterate processing activities at a time frequency of at least once per second. The processor may iterate processing activities at a spatial frequency of at least once per 1m. The processor may iterate processing activities at or before a bifurcation point in a given trajectory. The processor may iterate processing activities at or before a merge point in a given trajectory. The processor may iterate processing activities at or before an intersection point in a given trajectory.

[0030] Implementations of the device may include a display, or communicate with a display to visualize determined speed behaviors.

[0031] These and other aspects, features, and implementations may be represented as a method, device, system, component, program product, method of doing business, parts or steps for performing a function, and in other ways.

[0032] These and other aspects, features, and implementations will become apparent from the following description that includes the claims. Description of the Drawings

[0033] Figure 1 is a block diagram of an AV system.

[0034] Figure 2 Shows an example of a speed behavior planner communicating with other devices.

[0035] Figure 3 Shows an example of speed behavior planning processing.

[0036] Figure 4 Shows examples of speed behavior factors and their speed behaviors.

[0037] Figures 5 to 12 Shows examples of candidate speed behaviors and determined speed behaviors.

[0038] Figure 13 Shows an example of the processing flow of a speed behavior planner. Detailed Description

[0039] The term "autonomous vehicle" or "AV" is used broadly to include, for example, vehicles having one or more autonomous driving capabilities.

[0040] The term "autonomous driving capability" is used broadly to include, for example, any function, feature, or facility that can participate in driving an AV other than by a person manipulating the steering wheel, accelerator, brakes, or other physical controllers of the AV.

[0041] The term "trajectory" is used broadly to include, for example, any path or route from one place to another; for example, a path from a pick-up location to a drop-off location, or a path towards a destination location. In some implementations, a trajectory can be combined with speed behavior information.

[0042] The term "destination" or "destination location" is used broadly to include, for example, any place that the AV is to reach, including, for example, a temporary drop-off location, a final drop-off location, or a destination, etc.

[0043] The term "driving environment" is used broadly to include, for example, any characteristic, property, condition, or parameter of the physical world in which the vehicle is traveling, including, by way of example, a road network, and static and moving physical objects such as buildings, other vehicles, or pedestrians. The driving environment can be associated with the closest or nearby or adjacent, or with a more distant place such as a place along the planned trajectory of the vehicle.

[0044] The term "event" is used broadly to include, for example, any event that can interfere with traveling along a road network, such as, for example, a sports event, a game, a marathon, a concert, a fire, a flood, a collision, a traffic light failure, or bad weather.

[0045] The term "speed behavior factor" is used broadly to include, for example, any feature, property, parameter, environment, influence, value, object, context, specification, condition, law, or constraint used to derive one or more speed behaviors or related to one or more speed behaviors, or applied or applicable to calculate, propose, control, suggest, or indicate one or more speed behaviors.

[0046] The term "speed behavior" is used broadly to include, for example, any behavior or operation of a vehicle that includes, defines, implies, is based on, or otherwise relates to the speed of the vehicle, such as a constant speed, a maximum speed, a minimum speed, a changing speed, a zero speed, a derivative of speed, a mathematical function of speed, a drivable speed pattern, or any other speed profile. In some cases, a speed behavior can be derived, proposed, controlled, suggested, or indicated computationally or in some other way from one or more speed behavior factors or information related to the driving environment or vehicle system conditions, or a combination thereof. A speed behavior can be expressed as a function in the time domain or the spatial domain or both domains. A speed behavior can be expressed at a given time or during a time period, including any fixed speed, changing speed, minimum speed, maximum speed, changing speed profile, conditional speed profile, candidate speed profile, determined speed profile, or a combination thereof. A speed behavior may involve speed limits, such as a maximum speed or a minimum speed suggested or required by the speed behavior, etc.

[0047] The term "speed limit" is used broadly to include, for example, a speed behavior that includes a limit on the speed of a vehicle, such as a maximum speed limit or a minimum speed limit, etc.

[0048] The term "candidate speed behavior" is used broadly to include, for example, any assumed, possible, potential, or hypothetical speed behavior of a vehicle that is considered as part or all of the determined speed behavior of the vehicle.

[0049] The term "determined speed behavior" is used broadly to include, for example, any actual, decided, selected, or applied speed behavior used for or by a vehicle during vehicle operation. A determined speed behavior can be the same as a candidate speed behavior, derived from a candidate speed behavior, or a modified version of a candidate speed behavior.

[0050] AV system

[0051] This document describes techniques applicable to any vehicle having, for example, one or more automated driving capabilities, including fully automated vehicles, highly automated vehicles, and conditionally automated vehicles, such as so-called level 5, level 4, and level 3 vehicles, respectively (for more details on vehicle automation levels, see SAE International Standard J3016: Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems, the entire content of which is incorporated herein by reference). Automated driving capabilities can include controlling the steering or speed of the vehicle. The techniques described herein can also be applied to partially automated vehicles and driver assistance vehicles, such as so-called level 2 and level 1 vehicles (see SAE International Standard J3016: Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems). One or more of level 1, level 2, level 3, level 4, and level 5 vehicles can automate specific vehicle operations (e.g., steering, braking, and using maps) under specific driving conditions based on the processing of sensor inputs. The techniques described herein can benefit vehicles having any level of automation, ranging from fully automated vehicles to human-operated vehicles.

[0052] As Figure 1 shown, a typical activity of AV 100 is to travel safely and reliably, autonomously or partially manually or both, along a trajectory 198 through an environment 190 towards a destination location 199 while avoiding objects (e.g., mountain 191, vehicle 193, pedestrian 192, cyclist, and other obstacles) and complying with road rules (e.g., operating rules or driving preferences).

[0053] The driving of AV 100 is typically supported by a set of technologies (e.g., hardware, software, and stored real-time data), which are sometimes collectively (along with AV 100) referred to herein as AV system 120. In some implementations, one or some or all of the technologies are used on AV 100. In some cases, one or some or all of the technologies of the AV system can be used elsewhere, such as at a server (e.g., in a cloud computing infrastructure), etc. The components of AV system 120 can include one or more or all of the following components (and others).

[0054] 1. The functional device 101 of the AV 100, which is configured to receive and act on commands for driving (e.g., steering 102, accelerating, decelerating, gear selection, and braking 103) and for auxiliary functions (e.g., turn signal activation) from one or more computing processors 146 and 148.

[0055] 2. One or more data storage units 142 or memories 144 or both, for storing machine instructions or various types of data or both.

[0056] 3. One or more sensors 121, for measuring or inferring (or both) attributes of the AV state or condition, such as the position, linear and angular velocity and acceleration, and forward direction (e.g., the orientation of the front end of the AV). For example, such sensors can include, but are not limited to: GPS; an inertial measurement unit for measuring both vehicle linear acceleration and angular rate; individual wheel speed sensors for measuring or estimating individual wheel slip rate; individual wheel brake pressure or braking torque sensors; engine torque or individual wheel torque sensors; and steering wheel angle and angular rate sensors.

[0057] 4. One or more sensors for sensing or measuring attributes of the AV environment. For example, such sensors can include, but are not limited to: monocular or stereo cameras 122 operating in the visible, infrared, or thermal (or both) spectra; lidar 123; radar; ultrasonic sensors; time-of-flight (TOF) depth sensors; speed sensors; and temperature and rain sensors.

[0058] 5. One or more communication devices 140, for communicating attributes of the measured or inferred or both AV or other vehicle states and conditions, such as position, linear and angular velocity, linear and angular acceleration, and linear and angular forward direction, etc. These devices include vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication devices, and devices for wireless communication via point-to-point or ad hoc networks or both. The communication device 140 can communicate via the electromagnetic spectrum (including radio and optical communication) or other media (e.g., air and acoustic media).

[0059] 6. One or more communication interfaces 140 (e.g., wired, wireless, WiMAX, Wi-Fi, Bluetooth, satellite, cellular, optical, near field, or radio or combinations thereof), for sending data to and receiving data from a remote database 134, sending sensor data or data related to driving performance to or from the remote database 134, or sending information related to the operation or remote operation of the AV system, or combinations thereof.

[0060] 7. One or more data sources 142 for providing historical, or real-time or predictive information, or a combination of any two or more thereof, related to the AV environment 190, including, for example, maps, driving performance, traffic congestion updates, or weather conditions. Such data may be stored on the data storage unit 142 or the memory 144 of the AV 100, or may be sent to the AV 100 from a remote database 134 via a communication channel, or a combination thereof.

[0061] 8. One or more data sources 136 for providing digital road map data from a GIS database, potentially including one or more of the following: a high-precision map of road geometric attributes; a map describing the connectivity attributes of a road network; a map describing the physical attributes of a road (e.g., traffic speed, traffic volume, number of lanes for vehicles and cyclists, lane width, lane traffic direction, or lane marking type and location, or a combination thereof); and a map describing the spatial location characteristics of a road (e.g., crosswalks, traffic signs, or various other travel signals). Such data may be stored on the memory 144 of the AV 100, or may be sent to the AV 100 from a remote database server 134 via a communication channel, or a combination of both.

[0062] 9. One or more data sources 134 or sensors 132 for providing historical information related to the driving attributes (e.g., speed and acceleration behavior) of vehicles that have previously traveled along a local road segment at a similar time of day. Such data may be stored on the memory 144 of the AV 100, or may be sent to the AV 100 from a remote database server 134 via a communication channel, or a combination of both.

[0063] 10. One or more computing devices 146 and 148, located on the AV 100 (or remotely or both), for performing algorithms for (e.g., online) generating control actions based on real-time sensor data and prior information, thereby allowing the AV system 120 to perform its autonomous driving capabilities.

[0064] 11. One or more processes for processing sensor data, perceiving the environment, understanding the current and future possible conditions presented by the perceived environment, performing trajectory planning, performing motion control, and making decisions based on these perceptions and understandings. The processes may be implemented by integrated circuits, field-programmable gate arrays, hardware, software, or firmware, or a combination of two or more thereof.

[0065] 12. One or more interface devices 150 (e.g., a display, a mouse, a trackball, a keyboard, a touch screen, a biometric reader, augmented reality glasses, a gesture reader, a speaker, a microphone, and a recorder), which are connected to computing devices 146 and 148 to provide information and issue alerts to a user (e.g., an occupant or a remote user) of the AV 100 and receive input from the user. The connection can be wireless or wired. Any two or more interface devices can be integrated into a single device.

[0066] 13. One or more applications, which run on a computing device (e.g., a mobile device, a laptop computer, a tablet, or a smart phone) of a user of the AV system, for providing an interactive user interface on the computing device, communicating with the AV system (including with the AV), performing processing related to user participation in controlling the AV system, calls of the AV, use of the AV, providing information about the AV to the user, and various other functions.

[0067] Speed behavior factors

[0068] When an AV drives a vehicle on a road through an AV system, the speed behavior of the AV can be constrained, controlled, modulated, influenced, or changed by one or a combination of two or more speed behavior factors. Examples of speed behavior factors include the following:

[0069] 1. Legal regulations. Speed behaviors such as speed limits (maximum limit or minimum limit) can be specified by laws used as speed behavior factors. For example, the maximum speed limit on a highway section can be 65 miles per hour (mph); the speed limits of different highway sections may be different. The maximum speed limit on a downtown road can be 15 mph. In some instances, the law may require a vehicle to stop at a stop sign or a stop signal (e.g., a red light, a road sign, a stop line, or a gesture of a traffic controller, etc.), so the speed must be 0 mph before the stop sign or the stop signal. In some cases, a minimum speed limit can be set, for example, a minimum speed of 45 mph on a restricted access highway. In some cases, the law may impose other types of speed behaviors, such as "mountain speed through a tunnel" or "slow down in a work zone", etc. Some legal regulations can be regarded as "hard speed behavior factors" because these legal regulations include static specific statements of the required speed behaviors. Some legal regulations can be regarded as "conditional speed behavior factors" because these legal regulations state speed behaviors for their triggering conditions. For example, "If a vehicle is driving on an interstate highway outside the urban area, the maximum speed is 65 mph. If a vehicle is driving on an interstate highway in the urban area, the maximum speed is 55 mph." Some legal regulations are examples of "unconditional speed behavior factors", such as "The speed of a vehicle at a stop sign must be 0 mph", etc.

[0070] 2. Objects in the driving environment. When an object is detected in the driving environment, the presence of the object may require safety protection of the AV system or other appropriate speed behavior. For example, when approaching a crosswalk, the AV system may reduce its speed or come to a complete stop to yield to any pedestrians. In some implementations, pedestrians or animals may jaywalk, and the AV system may reduce its speed or come to a complete stop to avoid hitting a pedestrian or animal. In some instances, when driving on a road, the speed behavior of the AV system may require observing speed limits based on the speed of the vehicle ahead. In some implementations, the AV system may reduce its speed to avoid rubbing against or causing nervousness to neighboring objects (such as other vehicles, cyclists, or pedestrians walking, etc.). In some cases, the AV system may determine and then avoid a buffer zone near other objects (such as a parked sedan, a vehicle in an adjacent lane, or a neighboring person, etc.). For example, the buffer zone provides space for the unexpected movement of other objects. Generally, when the speed of the AV system is relatively high, a larger buffer zone may be preferred. When the speed is relatively low, a smaller buffer zone may be sufficient. In some cases, the available space near other objects will limit the possible size of the buffer zone and may constrain the AV system to reduce its speed to below the corresponding upper speed limit. For example, if the passable part of the street is narrowed due to snowdrifts, the AV system may slow down when passing along the narrowed passable part.

[0071] 3. Events. Events can occur near the AV system or can remain on the road as part of the planned trajectory of the AV system. Therefore, for example, a maximum speed limit can be imposed on the AV system based on the degree of congestion or traffic volume associated with the event.

[0072] 4. Mechanical limitations. Generally, for example, in cases where the AV is changing lanes, turning right, turning left, making a sharp turn, or making a U-turn, etc., the maneuver of the AV system will exert a lateral force on the AV. At these moments, the speed of the AV will have a longitudinal component along the direction of travel and a lateral component perpendicular to the direction of travel. To avoid rollover or skidding, the maneuvers that cause lateral forces (and the corresponding lateral component of the speed) must be coordinated with the longitudinal component of the speed, for example, by imposing a maximum speed limit on the longitudinal component. In some examples, a sudden change in speed (such as from 55 mph to 0 mph) is not applicable or appropriate for the mechanical system of the AV, so speed behavior can be imposed to enable the AV system to smoothly reduce its speed.

[0073] 5. AV component failure. If a component of the AV system fails during operation, the AV system can self-impose a maximum speed limit to reduce the risk associated with the component failure. For example, if the tire pressure drops below the safe level, the AV can self-impose a maximum speed limit of 45 mph.

[0074] 6. Weather conditions. The speed of the AV can be restricted by weather conditions. For example, a road surface that is wet or icy due to flooding, rain, or snow can cause the AV system to reduce its speed to not exceed the maximum speed limit to avoid risks. For example, during driving, the AV system may accidentally direct the vision sensor directly towards a strong light source (such as the sun), so the vision sensing may deteriorate due to extreme brightness; the deterioration of the sensing can cause the AV system to reduce its speed so that the AV system can safely respond to unexpected driving scenarios.

[0075] 7. User preferences. The AV system can be subject to speed limits imposed by the preferences of the user. For example, the driver of the AV system may prefer to keep its speed not exceeding a threshold, such as 40 mph. For example, a deliverer who uses the AV system to deliver fragile ceramic pieces may want to limit the speed of the AV system to avoid violent movements that may break the porcelain. In some cases, the AV system can be used to carry patients or the injured, and a maximum speed limit can be set to provide comfort to the patients.

[0076] 8. Road characteristics. Road characteristics can also cause speed limits. For example, when the curvature of the road is high, the maximum speed limit of the AV system can be lowered to avoid rollover or skidding. When the road surface is bumpy or not well-paved, the AV system can have a lower maximum speed limit. When the AV system is driving on a mountain road, the speed can be restricted to prevent driving off a cliff.

[0077] Speed Behavior Planner

[0078] As Figure 2 shown, the implementation of the AV system can include a speed behavior planner 201. An important activity of the speed behavior planner is to determine the speed behavior used by the AV based on various relevant information and generally impose a speed behavior on the driving of the AV. The speed behavior planner can be implemented by one or more of software, firmware, or hardware (such as a computing device, an electronic circuit, a field-programmable gate array, or an application-specific integrated circuit or a combination thereof, etc.), or a combination, or other means.

[0079] The speed behavior planner 201 can communicate with other components of the AV system 200 (such as receiving or sending information or instructions relative to other components of the AV system, etc.). In some cases, one or more of such components can be included as part of the speed behavior planner 201.

[0080] The speed behavior planner 201 can communicate with one or more sensors 202 and 204 (e.g., receive or send information or instructions relative to the sensors, etc.). Examples of sensors include: monocular cameras, stereo cameras, video cameras, lidars, radars, infrared sensors, ultraviolet sensors, thermometers, pressure sensors, odometers, speed sensors, and chemical sensors, etc. One or more sensors can provide signals representing the motion state of the AV system (e.g., longitudinal speed, lateral speed, acceleration or torque, or a combination thereof, etc.). In some cases, one or more sensors can provide signals related to the driving environment (e.g., another moving object on the road, traffic light signals, drivable areas, non-drivable areas, lanes, traffic volume, traffic speed, sidewalk markings or traffic signs, or a combination thereof, etc.).

[0081] The implementation of the speed behavior planner 201 can include communication with the motion planner 216 (e.g., receive or send information or instructions relative to the motion planner 216, etc.). The information received can include one or more trajectories towards the end position. In some examples, the speed behavior planner 201 can communicate with the perception processor 214 to receive information related to the perceived driving environment of the AV system 200 (e.g., traffic light signals, traffic volume, traffic speed, other vehicles, pedestrians, animals, events, and hidden objects, etc.). In some cases, the perception processor 214 can receive signals from the sensors, process the signals, and send the processing results to the speed behavior planner 201 for planning the speed behavior.

[0082] In some implementations, the speed behavior planner 201 can communicate with the local database 212 via a network interface or a data bus (e.g., receive or send information or instructions relative to the local database 212, etc.), or communicate with the remote database 230 via the wireless communication interface 210 (e.g., receive or send information or instructions relative to the remote database 230, etc.), or perform both communications. The database (212 or 230 or both) can store and provide various types of information. Examples of information include: event scheduling, static information (e.g., maps, road configurations, road complexities, buildings, traffic signs, traffic light positions, lanes, curbs, crosswalks, and sidewalks, etc.), dynamic information that can be perceived by the perception processor 214 (e.g., traffic light signals, traffic volume, traffic speed, other vehicles, pedestrians, animals, and events, etc.), and route information that can be created by the motion planner 216 (e.g., trajectories towards the end position, etc.).

[0083] In some implementations, the speed behavior planner 201 can communicate with the user interface 234 (e.g., receive or send information, instructions, etc. relative to the user interface 234), and the user interface 234 can be on a computing device (such as a smart phone, mobile device, portable device, or desktop computer, etc.). The user interface 234 can be presented locally within or outside the AV, or at a remote location, or a combination thereof. There can be two or more user interfaces that communicate with the speed behavior planner 201 simultaneously. The user interface 234 can display speed information (such as the speed of the AV system, the speed of other vehicles, speed limits, speed behavior factors, or perceived objects, or a combination thereof). In some cases, the communication with the user interface 234 can occur via a local network on the AV system 200. In some implementations, the communication with the user interface 234 can occur via the server 232. In addition to presenting information to the user, the user interface can also receive preferences, instructions, or other information from the user, such as information related to preferences or desired speed limits or other speed behaviors, etc. On a mobile phone or other mobile device, the user interface can be presented through a local application installed on and running on the mobile device. The local application can be considered part of the AV system.

[0084] In some implementations, the speed behavior planner 201 can communicate with the server 232 (e.g., receive or send information, instructions, etc. relative to the server 232), and the server 232 can be located on the AV 200, at a remote location, or at both. There can be two or more servers that communicate with the speed behavior planner 201 simultaneously. The server 232 can provide various types of services. For example, the server 232 can provide a data service by transmitting various types of the aforementioned information and data to the speed behavior planner 201. For example, the server 232 can transmit speed limits, speed behavior factors, speed behaviors, or other speed behaviors, or a combination thereof, to the speed behavior planner 201.

[0085] In some examples, the server 232 can provide a remote operation service, where a remote operator (such as a person, a computer program, or both) can transmit commands used for speed behavior planning to the speed behavior planner 201. (Additional information related to the remote operation service is included in U.S. Patent Applications 15 / 624,780, 15 / 624,802, 15 / 624,819, 15 / 624,838, 15 / 624,839, and 15 / 624,857, the entire contents of which are incorporated herein by reference.)

[0086] In some instances, server 232 may provide a fleet management service, where a fleet manager (e.g., a person or a computer program or both) may transmit fleet information to AV system 200 so that speed behavior planner 201 can make appropriate adjustments to speed behavior planning. For example, the fleet manager may specify a new destination location for the AV system, so the AV system can change its trajectory and adjust its speed behavior. In some implementations, the fleet manager may receive or predict new driving information resulting from the occurrence of an event, and the fleet manager may request the AV system to set a new speed limit in its speed behavior.

[0087] Speed behavior planning

[0088] The operation of the speed behavior planner (i.e., speed behavior planning) includes multiple processes. Referring to Figure 3 , the speed behavior planner 300 may receive one or more trajectories towards the destination location 312. In some implementations, the speed behavior planner 300 may receive driving environment information 314, such as a map, static information, dynamic information, or a combination thereof. For each received trajectory, the speed behavior planner 300 may identify speed behavior factors 322 and their associated candidate speed behaviors (e.g., speed limits, etc.) 324.

[0089] Generally, the speed behavior planner 300 may perform a computational analysis on the speed behavior factors or candidate speed behaviors or both to derive a determined speed behavior 326. The details of the derivation will be described below.

[0090] Candidate speed behaviors (e.g., speed limits, etc.) specified by hard speed behavior factors (e.g., legal speed limits on the road) may be regarded as hard maximum speeds. The determined speed behavior determined by the speed behavior planner 300 may be lower than the maximum speed specified by the hard speed behavior factors.

[0091] The implementation of the speed behavior planner 300 may include outputting candidate speed behaviors or determined speed behaviors. In some examples, the speed behavior may be a simple numerical speed value such as 65 mph. In some cases, an optimization process based on speed behavior factors of the AV system (e.g., mechanical performance or driving environment or both) may be used to computationally determine the speed behavior. The details of the computational analysis will be described below.

[0092] In some cases, identifying available speed behavior factors and evaluating the corresponding speed behaviors may be combined in a single step. The output of the speed behavior planner may include a speed behavior 340 that will be used to cause the controller of the AV to manipulate the AV according to the speed behavior.

[0093] Figure 4Show an example. The speed behavior planner can identify speed behavior factors and estimate the possible impacts of each speed behavior factor on speed behavior. When the AV system 400 travels along a road, there may be a maximum speed limit specified by a sign 410 or indicated in a road database for that section of the road. The perception processor of the AV system 400 can detect a cyclist 412 riding in front of the AV system 400, so the maximum speed limit of the AV can be constrained by the speed of the cyclist (e.g., 12 mph). There may also be a work zone 414 along the road where the road becomes narrower; the AV system 400 senses the work zone and the speed behavior planner estimates speed behavior factors to arrive at a candidate speed behavior that includes an appropriate maximum speed limit (e.g., 30 mph). The three maximum speed limits can be visualized by a plot 450, where line 452 represents the legal maximum speed limit of 55 mph, bar 454 represents the speed of the cyclist of 12 mph, and bar 456 corresponds to the appropriate maximum speed limit of 30 mph analyzed for the work zone.

[0094] Candidate speed behaviors in the form of maximum speed limits can be applied to long sections or short sections or individual locations. For example, in the speed - distance plot 450, the legal maximum speed limit 452 is applied to the entire road. However, the maximum speed limit 454 imposed by the cyclist and the maximum speed limit 456 implied by the work zone are only applied to the sections near them. Considering all the speed limits 452, 454, and 456, the speed behavior planner can derive a candidate speed behavior by selecting the minimum of the two maximum speed limits available at each location along the road, which results in the aggregated candidate speed behavior 462 shown in the plot 460. The candidate speed behavior 462 is not a constant speed but a contour of different speeds.

[0095] In some implementations, the plots 450 and 460 can be presented as speed - time by transforming the speed in the spatial domain to the time domain. This transformation can be based on the equation: distance = speed × time.

[0096] The candidate speed behavior can be adjusted in various ways to form a determined speed behavior. In some implementations, the speed behavior planner can derive a determined speed behavior from the candidate speed behavior by smoothing the discontinuities in the speed. Consider two types of discontinuities (step - up and step - down), and start with addressing the step - up discontinuity. Refer again to Figure 4, when the speed behavior factor no longer applies (e.g., when the AV system 400 overtakes the cyclist 412 at S2), the maximum upper speed limit of the AV system can be increased. For example, the plotted graph 460 shown illustrates a step-up discontinuity at position S2. Literally, a step-up discontinuity would require infinite acceleration, which is mechanically impossible. However, the fact that the AV cannot reach a higher maximum speed limit immediately after the step-up discontinuity does not pose a danger because any speed below the higher maximum speed limit is considered safe. Therefore, in the absence of safety concerns, the speed behavior planner may or may not smooth the step-up discontinuity. If smoothing is performed, the smoothing can be done using any of a variety of functions (e.g., linear function, non-linear function, quadratic function, third-order or higher-order function, Sigmoid function, hyperbolic function, logistic function, or other functions, or combinations thereof). Smoothing can also be done in a way that does not involve control by a specific mathematical function, but rather, for example, in response to changing behavioral factors and non-numerical behavioral factors. Reference Figure 5 , the speed behavior plotted graph 500 shows the linear interpolation speed behavior 502 at S2; the speed behavior plotted graph 510 shows the non-linear interpolation speed behavior 512.

[0097] Some implementations that adjust the candidate speed behavior to obtain the determined speed behavior can handle step-down discontinuities. Referring again to Figure 4 , if the AV system 400 travels at the legal maximum speed limit (55 mph) between positions S0 and S1, it will suddenly need to reduce its speed from 55 mph to 12 mph when it reaches S1. However, such a sudden speed reduction would require the AV system to apply infinite deceleration, which is mechanically impossible. Also, at position S1 (the actual position of the cyclist), if the AV system 400 cannot reduce its speed to the maximum speed limit imposed by the cyclist 412, the AV may collide with the cyclist. To avoid potential danger, the derivation of the determined speed behavior can include setting a smooth transition between speed limits, such as a transition that occurs before the time or position at which the lower upper speed limit is to be applied. Compared to the smoothing of step-up discontinuities, safety considerations may be more restrictive and more important in the smoothing of step-down discontinuities. For example, the derivation can use any of a variety of functions (e.g., linear function, non-linear function, quadratic function, third-order or higher-order function, Sigmoid function, hyperbolic function, logistic function, or other functions, or combinations thereof) to interpolate (including smoothing) between speed limits. Interpolation can also be done in a way that does not involve control by a specific mathematical function, but rather, for example, in response to changing behavioral factors and non-numerical behavioral factors. Reference Figure 6, the speed behavior plot 600 uses a linear interpolation scheme to obtain linear constraints 602 and 604; the plot 610 shows a non-linear interpolation scheme to obtain smoother constraints 612 and 614.

[0098] Deriving the speed behavior based on candidate speed behaviors can include considering the current driving speed of the AV system. For example, referring to plot 620, although the maximum available speed limit between S0 and S1 is 50 mph (622), the AV system can travel at a speed of 40 mph (624). Since encountering the next speed limit of 12 mph (628) may directly force the AV to decelerate, the smoothing of the step-down discontinuity at S1 may consider the current speed of 40 mph (624) rather than the available maximum speed limit of 50 mph (622) to produce the interpolated speed behavior 626.

[0099] Deriving the speed behavior can include considering the distance (or time) associated with a speed behavior factor, or a speed limit specified by a speed behavior factor, or a speed or a combination thereof specified by a speed behavior factor.

[0100] · For example, Figure 7 the plot 710 in shows the current position S70 of the AV system, a nearer speed behavior factor S71, and a farther speed behavior factor S72. Since the speed behavior factor S71 is nearer, the AV system may require a faster deceleration to reach the maximum speed limit of the speed behavior factor S71 compared to reaching the maximum speed limit of the speed behavior factor S72.

[0101] · For example, Figure 7 the plot 720 in shows two different maximum speed limits at position S71. The candidate speed behavior 722 with a higher upper speed limit can have a slower deceleration compared to the candidate speed behavior 724 with a lower upper speed limit.

[0102] · A moving object in the driving environment can become a speed behavior factor. Figure 7The plotted graph 730 in [description] shows an example. The AV system can be traveling (e.g., at 50 mph) at position S70 and detect a moving object (e.g., another vehicle, cyclist, pedestrian, animal, or toy, etc.) at S71 with a speed of, for example, 35 mph. The speed behavior planner can determine a maximum speed limit (e.g., 35 mph) based on the speed of the moving object and generate a determined speed behavior 732. Since both the AV system and the moving object are in motion, the speed behavior planner can continue to monitor the speed of the moving object. For example, as indicated by the dashed line 734, the speed of the moving object may change. Later, when the AV system slows down (e.g., to 45 mph) and reaches position S75, the moving object may be at S76 and have a speed of, for example, 40 mph. Due to the change in the speed of the moving object, the AV system can adjust the determined speed behavior by changing the maximum speed limit to produce a new determined speed behavior 736. The speed behavior planner can select the new determined speed behavior 736 and discard the old determined speed behavior 732; in some cases, the old determined speed behavior 732 can be prioritized (e.g., due to safety considerations), and the new determined speed behavior 736 can be discarded.

[0103] In some implementations, the AV system can encounter two or more speed behavior factors within a road segment. For example, in Figure 8 as the AV system 800 approaches the vehicle 802 ahead, as shown by the candidate speed behavior 852 in the plotted graph 850, the AV system 800 needs to slow down to the speed of the vehicle 802 ahead or below the speed of the vehicle 802 ahead. At the same time, the AV system 800 can detect a red light 804, which will require the AV to stop as shown by the candidate speed behavior 854. As shown in the plotted graph 850, both candidate speed behaviors 852 and 854 can be applied simultaneously (e.g., superimposed) between positions S80 and S81. When setting the determined speed behavior, the speed behavior planner can select between the two candidate speed behaviors. For safety considerations, the candidate speed behavior with the lower speed (e.g., Figure 8 854 in [description]) can override other candidate speed behaviors.

[0104] In some implementations, two or more candidate speed behaviors can cross. For example, referring to Figure 9, a given trajectory can cause the AV system 900 to turn right, which may cause the candidate speed behavior 954 to slow down the AV system to a lower speed. At the same time, another vehicle 904 may appear, causing the speed behavior planner to generate another candidate speed behavior 952. To avoid a collision with vehicle 904, the candidate speed behavior 952 can constrain the AV system 900 to decelerate rapidly. Since the two candidate speed behaviors cross at position S91, the speed behavior planner can aggregate the two candidate speed behaviors (e.g., form a synthesis of the candidate speed behaviors) by selecting the section with the lower speed of the two candidate speed behaviors; that is, between S90 and S91, the candidate speed behavior 954 can take precedence over other candidate speed behaviors, and between S91 and S92, the candidate speed behavior 952 can take precedence over other candidate speed behaviors, resulting in the aggregated determined speed behavior 962 shown in the plot 960.

[0105] Generally, the speed behavior planner can perform various operations to form a determined speed behavior based on two or more other candidate speed behaviors at a given position that will be applied to a position section.

[0106] In some implementations, the behavior planning process can include queuing processing. Referring to the plot 950, when the AV system is at S90, the speed behavior planner can use a queue to hold the two candidate speed behaviors 952 and 954. As the AV system travels along the road, the speed behavior planner can check the current queue to optimally determine which candidate speed behavior or which parts of the candidate speed behaviors to use as the determined speed behavior based on criteria (such as which candidate speed behavior is the most constrained, etc.). For example, between S90 and S91, the speed behavior planner can check the queue to select the candidate speed behavior 954, and between S91 and S92, select the candidate speed behavior 952. Other criteria can be used to make the selection. And the queuing processing can be applied to more than two different candidate speed behaviors.

[0107] The implementation of the speed behavior planning process can include: generating candidate speed behaviors and determined speed behaviors that are available, appropriate, and can be complied with by the AV system. Although speed behavior derivation can remove discontinuities in the candidate speed behaviors, driving the AV system according to a given candidate speed behavior may cause discomfort or mechanical failures. For example, if the AV system travels at a speed that strictly follows the candidate speed behavior 962 in the plot 960, its deceleration will exhibit a discontinuity at position S91. Therefore, the speed behavior planner can generate a determined speed behavior by smoothing the acceleration and deceleration. For this purpose, the speed behavior planner can store the criteria and rules that define the characteristics of the available speed behaviors and apply them to form the determined speed behavior.

[0108] In some implementations, speed behavior planning may include an optimization algorithm. The optimization may search for a determined speed behavior for a road segment where there are multiple speed behavior factors. For example, discontinuities may exist across candidate speed behaviors, and an optimization algorithm may be invoked to remove the discontinuities. In some instances, in a road segment or time period, two or more candidate speed behaviors may cross, and thus an optimization algorithm may be invoked to identify the best speed behavior through the road segment.

[0109] The optimization may treat a speed behavior factor as a hard speed behavior factor or a soft speed behavior factor, or as a combination of a partial hard speed behavior factor and a partial soft speed behavior factor. For example, in Figure 10 Candidate speed behaviors (solid line 1002) are derived in plot 1000. In the case of treating a speed behavior as a hard speed behavior factor, the optimally generated determined speed behavior (e.g., dashed line 1004 or 1008) cannot exceed the hard speed behavior factor. For a speed behavior factor treated as a soft speed behavior factor, although the entire determined speed behavior is preferably located below the candidate speed behavior associated with the soft speed behavior factor, the best determined speed behavior (e.g., dashed line 1006) may partially exceed the candidate speed behavior.

[0110] The implementation of the optimization algorithm may relax a given speed behavior factor. In some cases, treating a speed behavior factor as a hard speed behavior factor may result in no solution in the optimization step. When there is no solution, the optimization algorithm may relax the hard speed behavior factor; for example, the speed behavior may instead be treated as a soft speed behavior factor. In some implementations, one or more portions of a speed behavior factor may be treated as soft speed behavior factors. In some applications, the entire speed behavior factor may be treated as a soft constraint.

[0111] The implementation of the optimization algorithm performed by the speed behavior planner may consider the global or local smoothness of speed, acceleration (i.e., the first derivative of speed with respect to time), jerk (i.e., the second derivative of speed with respect to time), jounce (i.e., the third derivative of speed with respect to time), snap (i.e., the fourth derivative of speed with respect to time), crackle (i.e., the fifth derivative of speed with respect to time), or higher-order derivatives of speed with respect to time, or a combination thereof.

[0112] The implementation of the optimization algorithm may search for a determined speed behavior close to a given speed behavior. For example, in Figure 10In this case, during the optimization process, speed behavior 1008 can be a better solution than speed behavior 1004. Similarly, speed behavior 1006 can be a better solution than speed behavior 1008 because speed behavior 1006 deviates less from speed behavior 1002 compared to speed behavior 1008. However, if the optimization imposes a hard constraint on the speed behavior, then speed behavior 1008 can be a better solution than speed behavior 1006.

[0113] The implementation of the optimization algorithm can consider minimizing the travel time, which can be expressed as a function of speed. For example, based on the relationship: distance = speed × time, speed and time are inversely proportional. Therefore, when comparing two candidate speed behaviors 1004 and 1008 in the plot 1000, candidate speed behavior 1008 takes less time to reach the end position S100 than candidate speed behavior 1004. Thus, candidate speed behavior 1008 can become the preferred determined speed behavior. In another example, the speed behavior can be presented in the time domain, such as Figure 10 the plot 1010 etc. Two candidate speed behaviors 1012 and 1014 can be generated. Candidate speed behavior 1012 can take more time to stop the AV system than candidate speed behavior 1014. When minimizing the travel time, candidate speed behavior 1014 is the preferred solution (determined speed behavior) in the optimization.

[0114] The implementation of the optimization algorithm can consider maximizing the travel distance, which can be expressed as a function of speed. When considering time as a constant variable, the relationship: distance = speed × time means that searching for a speed behavior close to the speed limit set by the speed behavior factor is equivalent to maximizing the travel distance. For example, for the time period during which the AV system is driven (e.g., 5 seconds), the optimization algorithm can maximize the distance to identify the best determined speed behavior. Referring to the plot 1010, for the time period before T100, maximizing the distance is to maximize the area under the curve of the speed behavior. Thus, candidate behavior 1012 is the preferred solution (determined speed behavior) in the optimization.

[0115] Another example is based on Figure 10 the plot 1020. The speed behavior factor at S100 may require the AV system to stop. Two candidate speed behaviors 1022 and 1024 can be generated. Candidate speed behavior 1022 can almost stop the AV system directly in front of position S100, while candidate speed behavior 1024 can cause the AV system to decelerate faster and stop before position S100. When maximizing the travel distance, candidate speed behavior 1022 is the preferred solution (determined speed behavior) in the optimization.

[0116] The implementation of the optimization algorithm can consider past speed behavior or previously generated determined speed behavior, or both of them. Figure 11 An example is shown. At time t1, the plot 1100 shows the speed behavior factor S110 that results in the determined speed behavior 1102 followed by the AV system. At a later time t2, as shown in the plot 1110, the AV system reaches the position S111; a new candidate speed behavior 1104 can be derived, and the speed behavior planner can be triggered to search for other optimal determined speed behaviors. In this example, the optimization can consider the current driving speed at the position S111 and generate a new determined speed behavior 1106. Starting from the position S111, the AV system can discard the old determined speed behavior 1102 and follow the new determined speed behavior 1106.

[0117] The implementation of the optimization algorithm can use linear programming, non - linear programming, or dynamic programming, or a combination of them.

[0118] In some implementations, the speed behavior planning can repeatedly perform the above tasks and processes; this repetition can occur in time or in space, or in both. The speed behavior planning process can discretize the time domain. Thus, the execution of the planning process can be carried out at least or at most at 1Hz, 2Hz, 3Hz, 4Hz, 5Hz, 10Hz, 15Hz, 20Hz, 30Hz, 40Hz, 50Hz, 60Hz, 70Hz, 80Hz, 90Hz, 100Hz, 200Hz, 300Hz, 400Hz, 500Hz, 600Hz, 700Hz, 800Hz, 900Hz, or 1kHz. Similarly, the speed behavior planning process can discretize the space domain. Referring Figure 12 , when considering the trajectory 1200, the speed behavior planning can discretize the trajectory 1200 and perform speed behavior planning for each of the discretized spatial points (e.g., 1204, 1206, and 1208). The identification of speed behavior factors can be performed each time the AV approaches the next 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 spatial points. These points obtained by discretization can be evenly spaced or randomly spaced, or both. The distance between two adjacent discretized points can be at least or at most 1cm, 5cm, 10cm, 20cm, 30cm, 40cm, 50cm, 60cm, 70cm, 80cm, 90cm, 1m, 5m, or 10m.

[0119] In some implementations, a speed behavior planner may plan the speed behavior of an AV system (e.g., search for candidate speed behaviors or determine speed behaviors or both) for a subsequent road segment (e.g., at least or at most 1 cm, 5 cm, 10 cm, 20 cm, 30 cm, 40 cm, 50 cm, 60 cm, 70 cm, 80 cm, 90 cm, 1 m, 5 m, or 10 m). In some implementations, a speed behavior planner may plan the speed behavior of an AV system (e.g., search for candidate speed behaviors or determine speed behaviors or both) for a subsequent time period (e.g., at least or at most 1 second, 2 seconds, 3 seconds, 4 seconds, 5 seconds, 6 seconds, 7 seconds, 8 seconds, 9 seconds, 10 seconds, 20 seconds, 30 seconds, 40 seconds, 50 seconds, 60 seconds, 70 seconds, 80 seconds, 90 seconds, 100 seconds, 110 seconds, or 120 seconds). Given the planned speed behavior and the planned trajectory, the speed behavior planner may monitor whether the AV system follows the planned speed behavior on the planned trajectory. If there is a deviation, the motion controller of the AV system may include a feedback control mechanism to manipulate the AV system to follow the planned speed behavior or the planned trajectory or both.

[0120] In some implementations, a speed behavior planner may receive two or more trajectories from a motion planner. The two or more trajectories may be independent, parallel, intersecting, merging, or branching with each other, or a combination thereof. For example, Figure 12 shows two trajectories branching from trajectory 1220 at point 1204. The speed behavior planning may be performed for each of the two trajectories. In some cases, if the AV system selects one of the multiple trajectories (e.g., 1200) to travel along, the candidate and determined speed behaviors derived from the other trajectories (e.g., 1220) may be discarded. During this process, the speed behavior planner may consider one or more speed behaviors (which may include candidate speed behaviors or determined speed behaviors or both) along each of the given trajectories and send the speed behaviors to the motion planner. The motion planner may use the speed behaviors to optimally determine which trajectory is the preferred trajectory for the AV system to travel along. In other words, in some implementations, it is the motion planner rather than the speed behavior planner that finally selects the determined trajectory from two or more candidate trajectories.

[0121] In some implementations, speed behavior planning may include queuing processing. Referring to the plot 1250, assume that the trajectory is discretized and the discretization points are S121, S122, S123, and S124. When the AV system reaches S121, a new candidate speed behavior 1252 can be derived. When the AV system reaches S122, another new candidate speed behavior 1254 can be derived. The speed behavior planner may consider the most restricted candidate speed behavior (i.e., 1252) and discard the others. In some cases, instead of discarding the less restricted candidate speed behaviors, the speed behavior planner may place them in a queue for future use. For example, the AV system may reach a further position S123, at which a new candidate speed behavior 1256 is generated. The queue may store the candidate speed behaviors 1256, 1252, and 1254. The order of the speed behaviors in the queue may be random or metric-based. For example, the queue may sort the candidate speed behaviors from the lowest score (e.g., based on speed, time, distance, acceleration or jerk, or a combination thereof) to the highest score, or vice versa. In the plot of 1250, at S123, the queue may be configured to hold the candidate speed behaviors in the order of 1256, 1252, 1254, but may reorder the queue in the order of 1254, 1252, 1256 at position S124.

[0122] In some implementations, speed behavior planning may include optimizing the determined lateral speed behavior. The trajectory may include curved portions (e.g., curves, turns, or U-turns), such as the 1204 - 1206 - 1208 trajectory within the range where the speed behavior will have a lateral component. Any of the techniques disclosed in the present invention may be applicable to searching for the determined lateral speed behavior. Speed behavior planning may optimize the longitudinal and lateral speed behaviors together in an optimization solver, or may optimize the longitudinal and lateral speed behaviors in two different optimization solvers. Figure 12 In some implementations, the planning process may not always run at a specified time or space frequency. In some cases, the planning process may be triggered when a moving object is detected in the driving environment. Other speed behavior factors may include mechanical problems, special road features (e.g., uphill, downhill, potholes or curvatures, or a combination thereof), or special requests (e.g., hail, cyclist requests, server requests, remote operator requests, or a combination thereof) or a combination of them.

[0123]

[0124] Figure 13 ​Illustrates an exemplary processing flow of a speed behavior planner. The speed behavior planner may initialize a set of candidate speed behaviors (1302), and the set may be empty. The speed behavior planner may receive static driving information (e.g., from a map database 1308), or dynamic driving information (e.g., from a perception processor 1312), or both. The speed behavior planner may be given one or more planned trajectories (1304). In some cases, the given trajectory may be combined with static driving information, dynamic driving information, or both, combined with speed behavior factors, or combined with past or current or future speed behaviors, or a combination of them. For example, a trajectory planner may use a map to mark the locations of speed limits, traffic lights, traffic signs, or lane markings along a trajectory, which provides prior information for candidate speed behaviors. The speed behavior planner may use various information to generate candidate speed behaviors on a given trajectory (1310). In some implementations, the output may be sent to a trajectory planner 1316, and the trajectory planner 1316 may estimate the candidate speed behaviors to adjust the planned trajectory 1304. In some implementations, the output may be sent to an action controller 1314, and the action controller 1314 may control the AV system to drive along the planned trajectory at a speed following the speed behavior. The AV system may record the followed trajectory 1306, and the speed behavior planner may use the followed trajectory in the speed behavior planning process.

[0125] In the final step 1320, the result of the speed planning process may be transmitted to the next iteration and may become part of the initial set of candidate speed behaviors. The iteration continues until the AV system reaches the end position.

[0126] Other implementations (e.g., methods, software, mobile applications, operating systems, user interfaces, simulations, video games, hardware, electronic devices, global positioning systems, electronic circuits, field programmable gate arrays, or application specific integrated circuits, or combinations thereof) are also within the scope of the claims.

Claims

1. A method for operating a vehicle, comprising: identifying a speed limit along a trajectory by at least one processor; determining, by the at least one processor, a plurality of candidate speed profiles simultaneously applicable at a position on the trajectory by connecting the speed limits along the trajectory, wherein each candidate speed profile is a profile of a different speed along the trajectory; deriving, by the at least one processor, a determined speed profile by using the plurality of candidate speed profiles, wherein the derivation includes smoothing at least one speed discontinuity between two connected speed limits along the trajectory; and operating, by the at least one processor, a drive assembly of the vehicle according to the determined speed profile.

2. The method according to claim 1, wherein smoothing at least one speed discontinuity includes smoothing a step-down speed discontinuity.

3. The method according to claim 1, wherein smoothing at least one speed discontinuity includes smoothing a step-up speed discontinuity.

4. The method according to claim 1, wherein smoothing at least one speed discontinuity includes ignoring a step-up speed discontinuity.

5. The method according to claim 1, wherein deriving the determined speed profile includes using an optimization algorithm to find the determined speed profile.

6. The method according to claim 5, wherein deriving the determined speed profile includes treating the candidate speed profiles as soft constraints in the optimization algorithm.

7. The method according to claim 5, wherein deriving the determined speed profile includes treating the candidate speed profiles as hard constraints in the optimization algorithm.

8. A vehicle, comprising: a planning component for identifying a speed limit along a trajectory; determining a plurality of candidate speed profiles simultaneously applicable at a position on the trajectory by connecting the speed limits along the trajectory, wherein each candidate speed profile is a profile of a different speed along the trajectory; deriving a determined speed profile by using the plurality of candidate speed profiles, wherein the derivation includes smoothing at least one speed discontinuity between two connected speed limits along the trajectory; and operating at least one drive assembly of the vehicle according to the determined speed profile.

9. The vehicle according to claim 8, wherein smoothing at least one speed discontinuity includes smoothing a step-down speed discontinuity.

10. The vehicle according to claim 8, wherein smoothing at least one speed discontinuity includes smoothing a step-up speed discontinuity.

11. The vehicle according to claim 8, wherein smoothing at least one speed discontinuity includes ignoring a step-up speed discontinuity.

12. The vehicle according to claim 8, wherein the planning component finds the determined speed profile by deriving the determined speed profile by using an optimization algorithm.

13. The vehicle according to claim 12, wherein the planning component derives the determined speed profile by treating the candidate speed profiles as soft constraints in the optimization algorithm.

14. The vehicle according to claim 12, wherein The planning component derives the determined speed behavior by treating the candidate speed behaviors as hard constraints in the optimization algorithm.

15. An apparatus for operating a vehicle, comprising: a memory for storing instructions; and at least one processor for operating in accordance with the instructions when provided with the instructions, the operations including: identifying speed limits along a trajectory; determining a plurality of candidate speed behaviors simultaneously applicable at a position on the trajectory by connecting the speed limits along the trajectory, wherein each candidate speed behavior is a profile of a different speed along the trajectory; deriving a determined speed behavior by using the plurality of candidate speed behaviors, wherein the derivation includes smoothing at least one speed discontinuity between two connected speed limits along the trajectory; and operating a drive component of the vehicle in accordance with the determined speed behavior.

16. The apparatus according to claim 15, wherein smoothing at least one speed discontinuity includes smoothing a step-down speed discontinuity.

17. The apparatus according to claim 15, wherein smoothing at least one speed discontinuity includes smoothing a step-up speed discontinuity.

18. The apparatus according to claim 15, wherein smoothing at least one speed discontinuity includes ignoring a step-up speed discontinuity.

19. The apparatus according to claim 15, wherein the determined speed behavior is found by deriving the determined speed behavior by using an optimization algorithm.

20. The apparatus according to claim 19, wherein the determined speed behavior is derived by treating the candidate speed behaviors as partial soft constraints and partial hard constraints in the optimization algorithm.

21. A computer program product comprising a computer program which, when executed by a processor, implements the method for operating a vehicle according to any one of claims 1-7.

Images