Systems and methods for vehicles and computer-readable media

Abstract

This invention relates to systems and methods for vehicles and computer readable media. Techniques are described for identifying, by at least one processor of a vehicle, a reference path through an environment comprising a subset of a plurality of edges based on a graph comprising the plurality of edges and a plurality of nodes. The techniques further comprise identifying a first path based on an optimization of a spatial model related to the graph and the reference path and identifying a second path based on applying at least one constraint to the reference path. The techniques further comprise selecting, by the at least one processor, the first path or the second path as a path to be traversed by the vehicle based on a pre-identified rulebook. Other embodiments can be described or claimed.

Description

TECHNICAL FIELD

[0001] This specification relates to vehicle path planning. BACKGROUND

[0002] A vehicle, such as an autonomous vehicle, will utilize a path planning system to identify a path that the vehicle can navigate in an environment. In conventional systems, such a path planning system uses a sampling-based approach to identify the path. However, such a sampling-based approach can be inefficient, or can encounter inefficiencies when the speed of the vehicle is very low or near zero. Specifically, when the speed of the vehicle is very low or near zero, the time horizon in space will shrink, and the path planning system will be unable to efficiently produce a suggested path that exceeds the time horizon. Additionally, such a sampling-based approach can encounter difficulties when the path is constrained (e.g., due to the presence of other vehicles). SUMMARY

[0003] According to one aspect of the present invention, a system for a vehicle, wherein the system comprises: at least one processor; and at least one non-transitory computer- related medium comprising instructions that, when executed by the at least one processor, cause the at least one processor to: identify, based on a graph comprising a plurality of edges and a plurality of nodes, a reference path through an environment comprising a subset of the plurality of edges; identify, based on an optimization of a spatial model related to the graph and the reference path, a first path; identify, based on application of at least one constraint to the reference path, a second path; and select, based on a pre-identified rulebook, the first path or the second path as a path to be traversed by the vehicle.

[0004] According to another aspect of the present invention, a method for a vehicle, comprising: identifying, by at least one processor of the vehicle, based on a graph comprising a plurality of edges and a plurality of nodes, a reference path through an environment comprising a subset of the plurality of edges; identifying, by the at least one processor, based on an optimization of a spatial model related to the graph and the reference path, a first path; identifying, by the at least one processor, based on application of at least one constraint to the reference path, a second path; and selecting, by the at least one processor, based on a pre-identified rulebook, the first path or the second path as a path to be traversed by the vehicle.

[0005] According to yet another aspect of the present disclosure, one or more non-transitory computer-readable media comprising instructions that, when executed by at least one processor of a vehicle, cause the vehicle to: identify, based on a graph comprising a plurality of edges and a plurality of nodes, a reference path through an environment comprising a subset of the plurality of edges; identify a first path based on an optimization of a spatial model related to the graph and the reference path; identify a second path based on applying at least one constraint to the reference path; and select, based on a pre-identified rulebook, the first path or the second path as a path to be traversed by an autonomous vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 An example of an autonomous vehicle with autonomous capabilities is shown.

[0007] Figure 2 A computer system is shown.

[0008] Figure 3 An example architecture of an autonomous vehicle is shown.

[0009] Figure 4 A block diagram showing the relationship between inputs and outputs of a planning system is shown.

[0010] Figure 5 A directed graph used in path planning is shown.

[0011] Figure 6 An example path planning system according to embodiments is shown.

[0012] Figure 7 Other examples of directed graphs to be used in path planning according to embodiments are shown.

[0013] Figure 8 Example techniques for path planning according to embodiments are shown. DETAILED DESCRIPTION

[0014] In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, that the present disclosure can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily occluding the present disclosure.

[0015] In the drawings, specific arrangements or orders of illustrative elements (such as those representing devices, systems, instruction blocks, and data elements) are shown for ease of description. However, one skilled in the art will appreciate that the specific order or arrangement of illustrative elements in the drawings does not imply that a particular processing order or sequence must be followed, or that a process is necessarily separated into the separate blocks shown in the drawings. Also, inclusion of an illustrative element in a drawing does not imply that such element is required in all embodiments, nor that the features represented by such element cannot be included in or associated with other elements in some embodiments.

[0016] Also in the drawings, connecting elements, such as lines or arrows or the like, are used to illustrate connections, relationships or associations between two or more other illustrative elements, and the absence of such connecting elements does not imply the absence of a connection, relationship or association. In other words, connections, relationships or associations between some elements are not shown in the drawings to not obscure the disclosure with detail that is not necessary for understanding the concepts underlying the disclosure. Moreover, where multiple connecting elements are used to represent a connection, relationship or association between elements, one skilled in the art will appreciate that the use of a single connecting element to represent such a connection, relationship or association would be understood to illustrate the possibility of a single connection, relationship or association between elements, as opposed to multiple independent connections, relationships or associations between elements.

[0017] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description of embodiments, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to one skilled in the art that the various described embodiments can be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0018] Several of the described features can each be used independently of one another, or in any combination thereof. However, none of the individual features alone can solve any of the above-discussed problems, or can only solve one of the above-discussed problems. Some of the above-discussed problems can not be fully solved by any one of the features described herein. Although a title is provided, information related to a particular title can also be found elsewhere in the specification. Embodiments are described herein according to the following outline:

[0019] 1. OVERALL SUMMARY

[0020] 2. SYSTEM SUMMARY

[0021] 3. AUTONOMOUS VEHICLE ARCHITECTURE

[0022] 4. AUTONOMOUS VEHICLE PLANNING

[0023] 5. Path planning

[0024] SUMMARY

[0025] A vehicle (e.g., an autonomous vehicle) uses a path navigation system to navigate through an environment. Specifically, a path planning system identifies a reference path through the environment. A constraint computation system then applies at least one constraint to the reference path to identify one potential path through the environment. Additionally, a spatial model predictive control (MPC) system identifies other potential paths based on a spatial optimization of the reference path. The two potential paths are then compared to identify which path the vehicle should use to navigate. In embodiments, results from the spatial MPC are also fed back to the path planning system for identifying subsequent reference paths.

[0026] Some advantages of these techniques include improved ability of a vehicle to navigate through an environment with spatial constraints. Additionally, by allowing for the identification and comparison of two potential paths, the optimal path through the environment can be more quickly identified, thereby saving computational resources that would otherwise be used to identify the optimal path through the environment. Finally, because of the use of a speed and spatial constraint based approach, the navigation system is able to efficiently navigate through the environment even when the speed of the vehicle is low or near zero.

[0027] System Overview

[0028] Figure 1 An example of an autonomous vehicle 100 with autonomous capabilities is shown.

[0029] As used herein, the term "autonomous capabilities" refers to a function, feature, or facility that enables a vehicle to operate, in part or in whole, without real-time human intervention, including but not limited to fully autonomous vehicles, highly autonomous vehicles, and conditional autonomous vehicles. As used herein, an autonomous vehicle (AV) is a vehicle with autonomous capabilities.

[0030] As used herein, a "vehicle" includes a means of transporting goods or people. For example, a car, a bus, a train, an airplane, a drone, a truck, a boat, a ship, a submersible, a spacecraft, etc. A self-driving car is an example of a vehicle.

[0031] As used herein, a "trajectory" refers to a path or route that navigates an AV from a first spatiotemporal location to a second spatiotemporal location. In embodiments, the first spatiotemporal location is referred to as an initial location or a starting location, and the second spatiotemporal location is referred to as a destination, a final location, a target, a target location, or a target location. In some examples, a trajectory is composed of one or more segments (e.g., pieces of a road), and each segment is composed of one or more blocks (e.g., a portion of a lane or an intersection). In embodiments, a spatiotemporal location corresponds to a real-world location. For example, a spatiotemporal location is a pickup or drop-off location for a person or cargo.

[0032] As used herein, a "(one or more) sensor" includes one or more hardware components for detecting information related to the sensor's surrounding environment. Some hardware components can include sensing components (e.g., image sensors, biometric sensors), transmitting and / or receiving components (e.g., laser or radio frequency wave emitters and receivers), electronic components such as analog-to-digital converters, data storage devices such as random access memory (RAM) and / or non-volatile memory, software or firmware components, and data processing components such as application-specific integrated circuits, microprocessors, and / or microcontrollers.

[0033] As used herein, a "scene description" is a data structure (e.g., a list) or data stream that includes one or more classified or labeled objects detected by one or more sensors on an AV vehicle or provided by a source external to the AV.

[0034] As used herein, a "road" is a physical area that can be traversed by a vehicle, and can correspond to a named thoroughfare (e.g., a city street, an interstate highway, etc.) or can correspond to an unnamed thoroughfare (e.g., a driveway within a house or office building, a section of a parking lot, a section of an empty parking lot, a dirt path in a rural area, etc.). Because some vehicles (e.g., four-wheel drive pickup trucks, sport utility vehicles (SUVs), etc.) are capable of traversing a variety of physical areas that are not particularly well-suited for vehicle travel, a "road" can be any physical area that has not been formally defined as a thoroughfare by a municipality or other government or administrative body.

[0035] As used herein, a "lane" is a portion of a roadway that can be traversed by a vehicle. Sometimes a lane is identified based on lane markings. For example, a lane can correspond to most or all of the space between lane markings, or only a portion of the space between lane markings (e.g., less than 50%). For example, a roadway with lane markings far apart can accommodate two or more vehicles such that one vehicle can pass another without crossing a lane marking, and thus can be interpreted as a lane being narrower than the space between lane markings, or two lanes between lanes. Lanes can also be interpreted in the absence of lane markings. For example, a lane can be defined based on physical features of the environment (e.g., rocks and trees along a road in a rural area, or natural obstacles that should be avoided such as in underdeveloped areas). Lanes can also be interpreted independent of lane markings or physical features. For example, a lane can be interpreted based on an arbitrary path in an area that lacks obstacles that would otherwise be interpreted as features of a lane boundary. In an example scenario, an AV can interpret a lane through an unobstructed portion of a field or open space. In another example scenario, an AV can interpret a lane through a wide (e.g., wide enough for two or more lanes) roadway without lane markings. In this scenario, the AV can communicate information about the lane to other AVs so that the other AVs can use the same lane information to coordinate path planning between AVs.

[0036] The term "over-the-air (OTA) client" includes any AV, or any electronic device (e.g., computer, controller, IoT device, electronic control unit (ECU)) embedded in, coupled to, or in communication with an AV.

[0037] The term "OTA update" means any update, change, deletion, or addition to software, firmware, data, or configuration settings, or any combination thereof, delivered to an OTA client using proprietary and / or standardized wireless communication technologies, including but not limited to: cellular mobile communication (e.g., 2G, 3G, 4G, 5G), wireless radio area networks (e.g., WiFi), and / or satellite internet.

[0038] The term "edge node" refers to one or more edge devices coupled to a network that provide a portal for communication with AVs and can communicate with other edge nodes and cloud-based computing platforms to schedule and deliver OTA updates to OTA clients.

[0039] The term "edge device" refers to a device that implements an edge node and provides a physical wireless access point (AP) to a core network of an enterprise or service provider (e.g., VERIZON, AT&T). Examples of edge devices include, but are not limited to, a computer, a controller, a transmitter, a router, a routing switch, an integrated access device (IAD), a multiplexer, a metropolitan area network (MAN) and a wide area network (WAN) access device.

[0040] "one or more" includes a function performed by one element, a function performed by more than one element, e.g., in a distributed manner, a function performed by one element in a number of instances, a function performed by a number of elements in a number of instances, or any combination of the above.

[0041] It will also be understood that, although the terms "first," "second," etc. can be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the scope of the various described embodiments. The first contact and the second contact are both contacts, but they are not the same contact.

[0042] The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "comprises," "comprising," "includes," "including," "has," "having," "has" and / or "having," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0043] As used herein, the term "if' can be construed to mean "when" or "if when" depending on the context. Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be construed to mean "if when it is determined" or "if when [the stated condition or event] is detected," depending on the context.

[0044] As used herein, an AV system refers to an array of AVs and the hardware, software, stored data, and real-time generated data that support the operation of the AVs. In embodiments, the AV system is incorporated within the AVs. In embodiments, the AV system is distributed across several locations. For example, some software of the AV system is implemented on a cloud computing environment similar to a cloud computing environment.

[0045] In general, this document describes technology applicable to any vehicle with one or more levels of autonomy, including fully autonomous vehicles, highly autonomous vehicles, and conditionally autonomous vehicles, such as so-called Level 5, Level 4, and Level 3 vehicles, respectively (see SAE International Standard J3016: Taxonomy and Definitions for Terms Related to Driving Automation Systems for Road Vehicles, incorporated herein by reference in its entirety for more detailed information on levels of vehicle autonomy). The technology described in this document is also applicable to partially autonomous vehicles and driver-assist vehicles, such as so-called Level 2 and Level 1 vehicles (see SAE International Standard J3016: Taxonomy and Definitions for Terms Related to Driving Automation Systems for Road Vehicles). In embodiments, one or more Level 1, Level 2, Level 3, Level 4, and Level 5 vehicle systems can automatically perform certain vehicle operations (e.g., steering, braking, and use of maps) under certain operating conditions based on processing of sensor inputs. The technology described in this document can benefit vehicles at any level, ranging from fully autonomous vehicles to human-operated vehicles.

[0046] Autonomous vehicles have advantages over vehicles that require a human driver. One advantage is safety. From 1965 to 2015, the number of traffic fatalities per 100 million miles traveled in the United States decreased, in part due to additional safety measures deployed in vehicles. For example, warnings of an additional half-second associated with an impending collision are believed to mitigate many rear-end collisions. However, passive safety features (e.g., seat belts, airbags) can have reached their limit in improving this aspect. Thus, active safety measures such as automated control of a vehicle are a possible next step in improving these statistics. Since a human driver is believed to be the cause of a significant pre-crash event in most collisions, automated driving systems can achieve better safety outcomes by, for example: reliably recognizing and avoiding emergency situations better than a human; making better decisions than a human, following traffic laws better than a human, and predicting future events better than a human; and reliably controlling a vehicle better than a human.

[0047] Reference Figure 1The AV system 120 causes the vehicle 100 to operate along a trajectory 198 through the environment 190 to a destination 199 (sometimes referred to as a final location) while avoiding objects (e.g., natural obstacles 191, vehicles 193, pedestrians 192, cyclists, and other obstacles) and obeying road rules (e.g., operating rules or driving preferences).

[0048] In embodiments, the AV system 120 includes a device 101 for receiving and operating on operational commands from the computer processor 146. The term "operational command" is used to denote an executable instruction (or set of instructions) that causes the vehicle to take an action (e.g., a driving maneuver). Operational commands can include, without limitation, instructions for causing the vehicle to start moving forward, stop moving forward, start moving backward, stop moving backward, accelerate, decelerate, make a left turn, and make a right turn. In embodiments, the computer processor 146 is similar to the processor 204 described below with reference to FIG. 2. Examples of the device 101 include a steering controller 102, a brake 103, a gear, an accelerator pedal or other acceleration control mechanism, a windshield wiper, a side door lock, a window control, and a turn indicator. Figure 2

[0049] In embodiments, the AV system 120 includes sensors 121 for measuring or inferring properties of the state or condition of the vehicle 100, such as the AV's position, linear and angular velocities and linear and angular accelerations, and heading (e.g., the direction of the front end of the vehicle 100). Examples of the sensors 121 are a global positioning satellite (GPS), an inertial measurement unit (IMU) that measures both linear and angular rates of the vehicle, wheel rate sensors for measuring or estimating wheel slip, wheel brake pressure or brake torque sensors, engine torque or wheel torque sensors, and steering angle and angular rate sensors.

[0050] In embodiments, the sensors 121 also include sensors for sensing or measuring properties of the AV's environment. For example, monocular or stereo video cameras 122 in the visible, infrared, or thermal (or both) light spectrum, LiDAR 123, RADAR, ultrasonic sensors, time-of-flight (TOF) depth sensors, rate sensors, temperature sensors, humidity sensors, and precipitation sensors.

[0051] In embodiments, the AV system 120 includes a data storage unit 142 and a memory 144 for storing machine instructions associated with the computer processor 146 or data collected by the sensors 121. In embodiments, the data storage unit 142 is similar to the data storage unit 242 described below with reference to FIG. 2. Figure 2 ​The described ROM 208 or storage 210 are similar. In embodiments, the memory 144 is similar to the main memory 206 described below. In embodiments, the data storage unit 142 and the memory 144 store historical, real-time, and / or predictive information about the environment 190. In embodiments, the stored information includes maps, driving performance, traffic congestion updates, or weather conditions. In embodiments, data related to the environment 190 is transmitted from the remote database 134 to the vehicle 100 through a communication channel.

[0052] In embodiments, the AV system 120 includes communication devices 140 for transmitting properties of other vehicles' states and conditions, such as position, linear and angular velocity, linear and angular acceleration, and linear and angular heading, measured or inferred to the vehicle 100. These devices include vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication devices and devices for wireless communication through point-to-point or ad hoc networks or both. In embodiments, the communication devices 140 communicate across the electromagnetic spectrum, including radio and optical communications, or other media (e.g., air and acoustic media). The combination of vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) communication (and, in some embodiments, one or more other types of communication) is sometimes referred to as vehicle-to-everything (V2X) communication. V2X communication is generally in compliance with one or more communication standards for communication with and between autonomous vehicles.

[0053] In embodiments, the communication devices 140 include a communication interface. For example, a wired, wireless, WiMAX, Wi-Fi, Bluetooth, satellite, cellular, optical, near field, infrared, or radio interface. The communication interface transmits data from the remote database 134 to the AV system 120. In embodiments, the remote database 134 is embedded in a cloud computing environment. The communication devices 140 transmit data collected from the sensors 121 or other data related to the operation of the vehicle 100 to the remote database 134. In embodiments, the communication devices 140 transmit information related to teleoperation to the vehicle 100. In some embodiments, the vehicle 100 communicates with other remote (e.g., "cloud") servers 136.

[0054] In embodiments, the remote database 134 also stores and transmits digital data (e.g., stores data such as road and street locations). This data is stored in the memory 144 on the vehicle 100 or transmitted from the remote database 134 to the vehicle 100 through a communication channel.

[0055] In embodiments, the remote database 134 stores and transmits historical information (e.g., speed and acceleration profiles) related to driving attributes of vehicles that have previously traveled along the trajectory 198 at similar times of day. In one implementation, such data can be stored on the memory 144 on the vehicle 100, or transmitted from the remote database 134 to the vehicle 100 over a communication channel.

[0056] The computer processor 146 located on the vehicle 100 generates control actions algorithmically based on both real-time sensor data and a priori information, allowing the AV system 120 to perform its autonomous driving capabilities.

[0057] In embodiments, the AV system 120 includes computer peripherals 132 coupled to the computer processor 146 for providing information and alerts to a user of the vehicle 100 (e.g., an occupant or a remote user) and receiving input from the user. In embodiments, the peripherals 132 are similar to the display 212, input device 214, and cursor control 216 discussed below with reference to FIG. 2. The coupling is wireless or wired. Any two or more of the interface devices can be integrated into a single device. Figure 2

[0058] In embodiments, the AV system 120 receives and enforces a privacy level of an occupant, for example, specified by the occupant or stored in a profile associated with the occupant. The privacy level of the occupant determines how certain information associated with the occupant (e.g., occupant comfort data, biometric data, etc.) stored in the occupant profile and / or stored on the cloud server 136 and associated with the occupant profile is permitted to be used. In embodiments, the privacy level specifies certain information associated with the occupant that is deleted upon completion of a ride. In embodiments, the privacy level specifies certain information associated with the occupant and identifies one or more entities that are authorized to access the information. Examples of the specified entities that are authorized to access the information can include other AVs, third-party AV systems, or any entity that can potentially have access to the information.

[0059] The privacy level of the occupant can be specified at one or more levels of granularity. In embodiments, the privacy level identifies specific information to be stored or shared. In embodiments, the privacy level applies to all information associated with the occupant, such that the occupant can specify that her personal information is not stored or shared. The specification of entities that are permitted to access specific information can also be specified at various levels of granularity. Various sets of entities that are permitted to access specific information can include, for example, other AVs, the cloud server 136, specific third-party AV systems, etc.

[0060] ​In embodiments, the AV system 120 or the cloud server 136 determines whether the AV 100 or another entity has access to certain information associated with the occupant. For example, a third-party AV system attempting to access occupant input related to a particular spatiotemporal location must obtain authorization, e.g., from the AV system 120 or the cloud server 136, to access information associated with the occupant. For example, the AV system 120 uses the occupant’s specified privacy level to determine whether the occupant input related to the spatiotemporal location can be presented to the third-party AV system, the AV 100, or another AV. This enables the occupant’s privacy level to specify which other entities are allowed to receive data related to the occupant’s actions or other data associated with the occupant.

[0061] Figure 2 An example computer system 200 is illustrated in FIG. 2. In implementations, the computer system 200 is a special-purpose computing device. The special-purpose computing device is either hard-wired to perform the techniques, or includes digital electronic devices such as one or more application-specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are persistently programmed to perform the techniques, or one or more general purpose hardware processors programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such special-purpose computing devices can also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the techniques. In embodiments, the special-purpose computing device is a desktop computer system, portable computer system, handheld device, network device, or any other device that incorporates hard-wired and / or program logic to implement the techniques. In embodiments, the computer system 200 includes a bus 202 or other communication mechanism for communicating information, and a processor 204 coupled with the bus 202 for processing information. The processor 204 is, for example, a general-purpose microprocessor.

[0062] In embodiments, the computer system 200 also includes a read only memory (ROM) 208 or other static storage device coupled to the bus 202 for storing static information and instructions for the processor 204. A storage device 210, such as a magnetic disk, optical disk, solid-state drive, or three-dimensional crosspoint memory, is provided and coupled to the bus 202 for storing information and instructions.

[0063] In embodiments, the computer system 200 is coupled via the bus 202 to a display 212, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light-emitting diode (LED) display, or organic light-emitting diode (OLED) display for displaying information to a computer user. An input device 214, including alphanumeric and other keys, is coupled to the bus 202 for communicating information and command selections to the processor 204. Another type of user input device is a cursor control 216, such as a mouse, a trackball, a touchscreen, or cursor direction keys for communicating direction information and command selections to the processor 204 and for

[0064] According to one embodiment, the techniques herein are performed by the computer system 200 in response to the processor 204 executing one or more sequences of one or more instructions contained in the main memory 206. Such instructions can be read into the main memory 206 from another storage medium, such as the storage device 210. Execution of the sequences of instructions contained in the main memory 206 causes the processor 204 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry can be used in place of or in combination with software instructions.

[0065] The term “storage media” as used herein refers to any non-transitory media that store data and / or instructions that cause a machine to operate in a specific fashion. Such storage media include non-volatile media and / or volatile media. Non-volatile media include, for example, optical disks, magnetic disks, solid-state drives, or three-dimensional cross-point memory such as the storage device 210. Volatile media include dynamic memory, such as the main memory 206. Common forms of storage media include, for example, a floppy disk, a flexible disk, a hard disk, a solid- state drive, magnetic tape, or any other magnetic data storage medium, a CD-ROM, any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, an NV-RAM, or any other memory chip or cartridge.

[0066] Storage media are distinct from, but can be used in combination with, transmission media. Transmission media participate in transferring information between storage media. For example, transmission media include coaxial cables, copper wire, and fiber optic cables, including the wires that comprise the bus 202. Transmission media can also take the form of acoustic or light waves, such as those generated during radio frequency and infrared data communications.

[0067] In embodiments, various forms of media involve carrying one or more sequences of one or more instructions to the processor 204 for execution. For example, these instructions are initially carried on a disk or solid state drive of a remote computer. The remote computer loads the instructions into its dynamic memory and sends the instructions over a telephone line using a modem. A local modem of the computer system 200 receives the data on the telephone line and converts the data to an infrared signal using an infrared emitter. An infrared detector receives the data carried in the infrared signal and appropriate circuitry places the data on the bus 202. The bus 202 carries the data to the main memory 206, from which the processor 204 retrieves and executes the instructions. The instructions received by the main memory 206 can optionally be stored on the storage device 210 before or after execution by the processor 204.

[0068] The computer system 200 also includes a communication interface 218 coupled to bus 202. The communication interface 218 provides a two-way data communication coupling to a network link 220 that is connected to a local network 222. For example, the communication interface 218 is an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, the communication interface 218 is a local area network (LAN) card to provide a data communication connection to a compatible LAN. In some implementations, wireless links are also implemented. In any such implementation, the communication interface 218 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.

[0069] The network link 220 provides data communication through one or more networks to other data devices. For example, the network link 220 provides a connection to a host computer 224 or to cloud data centers or devices operated by an Internet Service Provider (ISP) 226 via a local network 222. The ISP 226 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 228. The local network 222 and the Internet 228 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link 220 and through the communication interface 218 are examples of transmission media for which the computer system 200 can be configured to use to carry digital data. In embodiments, the network 220 comprises a cloud or a portion of a cloud.

[0070] Computer system 200 communicates messages over the network(s), network link 220, and communication interface 218. In embodiments, computer system 200 receives code to be used to process data. The received code is executed by processor 204 as it is received, and / or stored in storage device 210, or other non-volatile storage for later execution.

[0071] Autonomous vehicle architecture

[0072] Figure 3 An example architecture 300 is shown for an autonomous vehicle (e.g., the vehicle 100 shown. Figure 1 The architecture 300 includes a perception system 302 (sometimes referred to as perception circuitry), a planning system 304 (sometimes referred to as planning circuitry), a control system 306 (sometimes referred to as control circuitry), a localization system 308 (sometimes referred to as localization circuitry), and a database system 310 (sometimes referred to as database circuitry). The systems each play a role in the operation of the vehicle 100. Collectively, the systems 302, 304, 306, 308, and 310 can be part of the AV system 120 shown. In some embodiments, any of the systems 302, 304, 306, 308, and 310 are a combination of computer software (e.g., executable code stored on a computer-readable medium) and computer hardware (e.g., one or more microprocessors, microcontrollers, application-specific integrated circuits [ASICs], hardware memory devices, other types of integrated circuits, other types of computer hardware, or a combination of any or all of these). Each of the systems 302, 304, 306, 308, and 310 is sometimes referred to as processing circuitry (e.g., computer hardware, computer software, or a combination of the two). A combination of any or all of the systems 302, 304, 306, 308, and 310 is also an example of processing circuitry. Figure 1

[0073] In use, the planning system 304 receives data representing a destination 312 and determines data representing a trajectory 314 (sometimes referred to as a route) that the vehicle 100 can travel in order to reach (e.g., arrive at) the destination 312. To cause the planning system 304 to determine the data representing the trajectory 314, the planning system 304 receives data from the perception system 302, the localization system 308, and the database system 310.

[0074] The perception system 302 uses, for example, also as described in the perception system 302, to determine data representing the environment 316 in which the vehicle 100 is located. Figure 1 ​The one or more sensors 121 shown identify nearby physical objects. The objects are classified (e.g., grouped into types such as pedestrian, bicycle, car, traffic sign, etc.) and a scene description including the classified objects 316 is provided to the planning system 304. The planning system 304 also receives data representing the AV position 318 from the localization system 308. The localization system 308 determines the AV position by using data from the sensors 121 and data from the database system 310 (e.g., geographic data) to compute a position. For example, the localization system 308 uses data from a GNSS (Global Navigation Satellite System) sensor and geographic data to compute the AV's longitude and latitude. In embodiments, the data used by the localization system 308 includes a high-precision map with lane geometry properties, a map describing road network connectivity properties, a map describing lane physical properties such as traffic speed, traffic volume, number of vehicle and bicycle lanes, lane width, lane traffic direction, or lane marking type and location, or a combination thereof, and a map describing spatial locations of road features such as intersections, traffic signs, or other types of travel signals. In embodiments, the high-precision map is constructed by adding data via automated or manual annotation to a low-precision map.

[0075] The control system 306 receives data representing the trajectory 314 and data representing the AV position 318 and operates the control functions 320a-c of the AV (e.g., steering, throttle, brakes, ignition) in a manner that will cause the vehicle 100 to travel the trajectory 314 to the destination 312. For example, if the trajectory 314 includes a left turn, the control system 306 will operate the control functions 320a-c in a manner that the steering angle of the steering function will cause the vehicle 100 to turn left and the throttle and brakes will cause the vehicle 100 to pause and wait for passing pedestrians or vehicles before making the turn.

[0076] Autonomous Vehicle Planning

[0077] Figure 4 The planning system 304 is shown to include a trajectory generator 302, a localization system 308, a database system 310, and a control system 306. The trajectory generator 302 receives data representing the destination 312 and generates a trajectory 314 that will cause the vehicle 100 to travel to the destination 312. The trajectory generator 302 receives data from the perception system 304 and uses the data to identify nearby physical objects. The trajectory generator 302 also receives data from the database system 310 and uses the data to identify nearby physical objects. The trajectory generator 302 uses the data from the perception system 304 and the database system 310 to generate a trajectory 314 that will cause the vehicle 100 to travel to the destination 312 while avoiding the nearby physical objects. Figure 3FIG. 4 is a block diagram 400 illustrating relationships between inputs and outputs of the planning system 304 (shown in FIG. 3). Generally, an output of the planning system 304 is a route 402 from a start point 404 (e.g., a source location or initial location) to an end point 406 (e.g., a destination or final location). The route 402 is typically defined by one or more route segments. For example, a route segment refers to a distance to be traveled over at least a portion of a street, road, highway, lane, or other physical area suitable for travel by a car. In some examples, the route 402 includes "off-road" route segments, such as an unpaved path or open field, for example, if the vehicle 100 is an off-road capable vehicle, such as a four-wheel drive (4WD) or all-wheel drive (AWD) car, SUV, or small truck.

[0078] In addition to the route 402, the planning system outputs lane-level route planning data 408. The lane-level route planning data 408 is used to travel the route segments of the route 402 based on conditions of the route segments at a particular time. For example, if the route 402 includes a multi-lane highway, the lane-level route planning data 408 includes trajectory planning data 410 that the vehicle 100 can use to select a lane from the multiple lanes, for example, based on whether an exit is proximate, whether one or more of the lanes have other vehicles in them, or other factors that change over the course of a few minutes or less. Similarly, in some implementations, the lane-level route planning data 408 includes speed constraints 412 specific to certain route segments of the route 402. For example, if the route segment includes pedestrians or unexpected traffic, the speed constraints 412 can limit the vehicle 100 to a slower travel speed than an expected speed, such as a speed based on speed limit data for the route segment.

[0079] In embodiments, inputs to the planning system 304 include (e.g., from the database system 310 (shown in FIG. 3), the database data 414, the current location data 416 (e.g., the AV location 318 (shown in FIG. 3)), the destination data 418 (e.g., for the destination 312 (shown in FIG. 3)), and the object data 420 (e.g., as described above with respect to FIG. 3). Figure 3 In embodiments, inputs to the planning system 304 include (e.g., from the database system 310 (shown in FIG. 3), the database data 414, the current location data 416 (e.g., the AV location 318 (shown in FIG. 3)), the destination data 418 (e.g., for the destination 312 (shown in FIG. 3)), and the object data 420 (e.g., as described above with respect to FIG. 3). Figure 3 In embodiments, inputs to the planning system 304 include (e.g., from the database system 310 (shown in FIG. 3), the database data 414, the current location data 416 (e.g., the AV location 318 (shown in FIG. 3)), the destination data 418 (e.g., for the destination 312 (shown in FIG. 3)), and the object data 420 (e.g., as described above with respect to FIG. 3). Figure 3 In embodiments, inputs to the planning system 304 include (e.g., from the database system 310 (shown in FIG. 3), the database data 414, the current location data 416 (e.g., the AV location 318 (shown in FIG. 3)), the destination data 418 (e.g., for the destination 312 (shown in FIG. 3)), and the object data 420 (e.g., as described above with respect to FIG. 3). Figure 3The perception system 302 shown perceives classified objects 316. In some embodiments, database data 414 includes rules used during planning. The rules are specified using a formal language (e.g., Boolean logic). At least some of these rules will apply to any given situation encountered by vehicle 100. A rule applies to a given situation if it has conditions satisfied based on information available to vehicle 100 (e.g., information about the surrounding environment). Rules can have priorities. For example, a rule “move to the leftmost lane if the road is a highway” can have a lower priority than “move to the rightmost lane if the exit is within a mile.”

[0080] Figure 5 This is illustrated in path planning (e.g., by planning system 304). Figure 3 The directed graph used is 500. Generally speaking, such as... Figure 5 The directed graph 500 shown is used to determine any path between a starting point 502 and an ending point 504. In the real world, the distance separating the starting point 502 and the ending point 504 may be relatively large (e.g., in two different urban areas) or relatively small (e.g., two intersections adjacent to a city block or two lanes of a multi-lane road).

[0081] In an embodiment, the directed graph 500 has nodes 506a-506d representing different locations that the vehicle 100 may occupy between the starting point 502 and the ending point 504. In some examples, for instance, when the starting point 502 and the ending point 504 represent different urban areas, nodes 506a-506d represent road segments. In some examples, for instance, when the starting point 502 and the ending point 504 represent different locations on the same road, nodes 506a-506d represent different locations on that road. Thus, the directed graph 500 includes information at different levels of granularity. In an embodiment, the directed graph with high granularity is also a subgraph of another directed graph with a larger scale. For example, a directed graph where the starting point 502 and the ending point 504 are far apart (e.g., many miles apart) has most of its information at a low granularity, and this directed graph is based on stored data, but it also includes some high-granularity information for representing a portion of the physical location in the field of view of the vehicle 100.

[0082] Nodes 506a-506d are different from objects 508a-508b, which cannot overlap with nodes. In embodiments, at low granularity, objects 508a-508b represent areas that a vehicle cannot traverse, e.g., areas without streets or roads. At high granularity, objects 508a-508b represent physical objects in the field of view of vehicle 100, e.g., other vehicles, pedestrians, or other entities with which vehicle 100 cannot share physical space. In embodiments, some or all of objects 508a-508b are static objects (e.g., objects that do not change position, such as streetlights or utility poles) or dynamic objects (e.g., objects that can change position, such as pedestrians or other cars).

[0083] Nodes 506a-506d are connected by edges 510a-510c. If two nodes 506a-506b are connected by edge 510a, vehicle 100 can travel between one node 506a and the other node 506b, e.g., without having to travel to an intermediate node before reaching the other node 506b. (When referring to vehicle 100 traveling between nodes, it means that vehicle 100 travels between two physical locations represented by the respective nodes.) Edges 510a-510c are typically bidirectional, in the sense that vehicle 100 can travel from a first node to a second node, or from the second node to the first node. In embodiments, edges 510a-510c are unidirectional, in the sense that vehicle 100 can travel from a first node to a second node, but vehicle 100 cannot travel from the second node to the first node. Edges 510a-510c are unidirectional in cases where they represent, e.g., one-way streets, individual lanes of a street, road, or highway, or other features that can only be traversed in one direction due to legal or physical constraints.

[0084] In embodiments, planning system 304 uses directed graph 500 to identify a path 512 consisting of nodes and edges between start node 502 and end node 504.

[0085] Edges 510a-c have associated costs 514a-b. Costs 514a-b are values that represent resources that will be spent if the vehicle 100 chooses that edge. A typical resource is time. For example, if one edge 510a represents a physical distance that is twice that of another edge 510b, the associated cost 514a of the first edge 510a can be twice that of the associated cost 514b of the second edge 510b. Other factors that affect time include expected traffic, number of intersections, speed limits, etc. Another typical resource is fuel economy. Two edges 510a-b can represent the same physical distance, but one edge 510a requires more fuel than the other edge 510b due to, for example, road conditions, expected weather, etc.

[0086] When the planning system 304 identifies a path 512 between the start point 502 and the end point 504, the planning system 304 typically chooses a path that is optimized for cost, e.g., the path that has the minimum total cost when the individual costs of the edges are added together.

[0087] Trajectory planning

[0088] As previously mentioned, purely sampling-based approaches for path planning, and in particular time-parameterized approaches, can be inefficient or can have difficulty in situations where the speed of the vehicle is very low or near zero or where the path is constrained. Embodiments herein provide a technique that utilizes a constraint computing system and an MPC system to alleviate these problems. The constraint computing system applies at least one constraint to a reference path to identify one potential path through the environment. The MPC system identifies other potential paths based on a spatial optimization of the reference path. The two potential paths are then compared to identify which path the vehicle should use for navigation.

[0089] Figure 6 An example path planning system according to embodiments is shown. In particular, Figure 6 A detailed example of the planning system 304 of Figure 3 is depicted. The planning system 304 includes a number of subsystems as depicted by Figure 6 The various subsystems can be implemented by hardware, software, firmware, or some combination thereof. In embodiments, the various subsystems of the planning system 304 are implemented in a single circuit, a single processor, a single processor core, etc. In other embodiments, one or more subsystems of the planning system 304 are implemented on a different circuit / processor / processor core than the other subsystems. It will also be noted that this depiction of the planning system 304 is intended as a high-level example embodiment for the purposes of discussion herein. Other embodiments can have a different number of elements or elements arranged in a different configuration than depicted by Figure 6 ​

[0090] The planning system 304 includes a sampling-based path planning system 605. The sampling-based path planning system 605 is used to generate a directed graph comprising a series of nodes and edges. Figure 7 An example of such a directed graph 700 to be used in path planning is shown in accordance with various embodiments. The graph 700 is used to identify a traversal path for a vehicle from a start point 705 (depicted in Figure 7 the middle of the graph 700) to an end point 730. As can be seen in Figure 7 the middle of the graph 700, the traversal of the vehicle from its start point 705 to the end point 730 is limited by factors such as lane markings 715 or obstacles 710 of a road. It will be understood that this graph 700 is intended as a highly simplified example, and other embodiments can include additional limitations or different types of limitations (e.g., cyclists, pedestrians, traffic-based limitations such as stop signs, etc.) or objects 508a or 508b.

[0091] The graph 700 is similar to the graph 500 of Figure 5 and shares one or more characteristics with the graph 500 of Figure 5 Specifically, the graph 700 includes a plurality of edges 720 and nodes 725, which are similar to and share one or more characteristics with the edges 510a-510c and nodes 506a-510d, respectively. Specifically, the edges 720 represent different driving trajectories between the nodes 725, which represent physical locations. The sampling-based path planning system 605 will also identify a reference path 735 through the graph 700. The reference path 735 is based on, for example, a sampling-based approach. As used herein, a “sampling-based approach” refers to an approach in which the sampling-based path planning system 605 samples points of the graph 700 until a path between the start point 705 and the end point 730 of the vehicle is found. In some embodiments, the sampling can be random or quasi-random, while in other embodiments, the sampling is guided by heuristics or done according to a specific set of motion priors. The sampling-based path planning system 605 outputs the graph 700, as well as the reference path 735 (or an indication thereof), to the constraint computation system 615 and the spatial trajectory optimization system 610 (which can also be referred to as a spatial MPC system).

[0092] The constraint computation system 615 is used to compute one or more constraints to be applied to the reference path 735. For example, the constraints include constraints such as speed-based constraints, such as maximum rates of the vehicle based on various conditions (traffic, weather, whether the vehicle is going straight or turning, etc.). Other examples of constraints include constraints based on various objects, such as the obstacle 710. For example, the constraints can involve spatial-based constraints, where the location of the vehicle is constrained to avoid collisions. Other examples of constraints include constraints such as lane constraints, imposed by lane markings 715, for example. To impose these constraints, the constraint computation system 615 can gather data from various sources, such as the map 700 or the reference path 735 provided by the sample-based path planning system 605, information provided by one or more sensors 121 described above (e.g., information from LiDAR systems, RADAR systems, cameras, etc.), information based on accelerometers or gyroscopes of the vehicle, GPS information, etc. The spatial trajectory optimization system 610 is configured to use as input one or both of the reference path 735 and the map 700 output by the path planning system 605. As described in further detail below, the spatial trajectory optimization system 610 is referred to as "gradient-based." The spatial trajectory optimization system 610 is then configured to identify an alternative path between the start point 705 and the end point 730. Specifically, to identify this alternative path, the spatial trajectory optimization system 610 is configured to consume data related to the reference path 735 and the map 700 into the spatial domain, rather than the classical time parameterization typically used for MPC.

[0093] Specifically, a model such as a spatial kinematic bicycle model is used to model the motion of the vehicle in the spatial domain. The spatial kinematic bicycle model is a model of the vehicle using site-based state variables, such as lateral error of the vehicle, local heading, velocity, acceleration, or steering angle, etc. In embodiments, the model also uses input variables such as changes in acceleration or changes in steering. Additionally, in embodiments, the model also uses variables related to slack in the system, such as slack on the lateral error, slack on the velocity, or slack on the acceleration, etc.

[0094] As used herein, the term "slack" refers to an acceptable level of constraint satisfaction. Specifically, a slack variable is an additional decision variable that an optimizer can change, which allows for some slack on the constraints of the optimization problem. Slack represents that some constraints can be violated at a certain cost. This ability to violate certain constraints allows, for example, for traffic rules / safety prioritization to be implemented.

[0095] The vehicle states are described in a local coordinate system, where variables are described relative to some reference path (this is specific to our implementation). The "lateral error" state represents the position of the vehicle in the lateral direction from the reference path.

[0096] It will be understood that these variables are described as example variables, and that other embodiments can use more or fewer variables, different variables, etc. to provide a spatial domain model of the behavior of the vehicle from the start point 705 to the end point 730.

[0097] The spatial trajectory optimization system 610 is configured to optimize the vehicle states over a prediction horizon to identify a second path, referred to herein as a "spatial-based path." For example, the optimization of the model is based on one or more of the following components:

[0098] A first component, which are the free variables for which the optimization seeks to find optimal values. These free variables are typically the vehicle states over a given prediction horizon.

[0099] A second component, which is the use of the vehicle model as an equality constraint. In other words, the evolution of the vehicle states over the prediction horizon is required to satisfy the vehicle model. The optimal sequence of vehicle states resulting from the optimization will thus obey the vehicle model.

[0100] A third component, which is an objective function that encodes a cost associated with certain combinations of the optimization variables. The objective function is generally pre-identified based on the desired behavior of the vehicle. Typically, the objective function involves one or more factors, such as passenger comfort, reference tracking, or clearance from obstacles, etc.

[0101] A fourth component, which are constraints that bound the free variables (as discussed above with respect to the first component) to a particular feasible set. These constraints describe the parameters in which the optimal solution must be found. The constraints model, for example, lane boundaries, collision constraints, speed constraints, actuation constraints (throttle, brake, or steering), etc. The details of the environment and the current vehicle state are then processed into an optimization problem for each iteration to be solved by the spatial trajectory optimization system 610. The solution to the optimization problem is a sequence of vehicle states that is optimal with respect to the objective function (e.g., the third component described above) subject to the constraints (e.g., the fourth component described above). Typically, this optimization technique is referred to as "gradient-based," as the objective function (e.g., the third component described above) is a smooth function representation that allows the model to move on the surface of the function and follow the gradient of the function to identify the optimal solution.

[0102] In one embodiment, it will be appreciated that the planning system 304 iterates according to a given time interval. That is, the planning system 304 updates one or more of the map 700, the reference path 735, etc. at a given frequency (which can be 1 hertz, for example). In other embodiments, the frequency can be higher or lower depending on factors such as the hardware used in the vehicle, the current state of traffic or weather, or other latency requirements. For this iteration, the space-based path can be provided to the sample-based path planning system 605 for use in identifying the next iteration of the reference path (or a portion thereof). In one embodiment, the sample-based path planning system 605 takes the space-based path as the reference path. In other embodiments, the sample-based path planning system 605 takes only a portion of the space-based path as the reference path, or uses the space-based path as a starting point for sampling. Other variations can be present in other embodiments.

[0103] The space-based paths are provided from the spatial trajectory optimization system 610 to the suggestion comparison system 620. Similarly, the constraint computation system 615 applies to the sampled reference paths 735, and the constraint computation system outputs constrained reference paths to the suggestion comparison system 620. The suggestion comparison system 620 receives the paths from the constraint computation system 615 and the spatial trajectory optimization system 610, and compares the paths. In particular, the constraint computation system 615 compares the constrained reference paths and the space-based paths using one or more rulebooks. The rulebooks relate to various factors, such as collision avoidance, road rules, etc. In particular, the rules are applied to the respective paths, and then a metric or value is generated based on the application. In an example, a rule relating to collision avoidance can be applied to the respective paths, and (a) for respective paths determined to have a collision, the generated metric or value can be zero (e.g., indicating a failure to comply with the rule), and (b) for respective paths determined to not have a collision, the generated metric or value can be non-zero (e.g., a value of 1 indicates that no collision will occur, and / or a value greater than 0 and less than 1 indicates a degree to which a collision can occur, etc.). In other examples, a rule relating to road rules (e.g., crossing a line dividing two lanes of opposing traffic) can be applied to the respective paths, and (a) for respective paths that comply with the road rule, the generated metric or value can be non-zero (e.g., indicating compliance with the rule, and / or a degree of compliance with the rule, etc.), and (b) for respective paths that do not comply with the road rule, the generated metric or value can be zero (e.g., indicating that it is not possible to comply with the rule for the given path). The metrics or values are then analyzed to identify which of the two provided paths most closely corresponds to the rules of the rulebook. In embodiments, the rulebook is pre-identified or includes pre-identified rules as described above. In other embodiments, one or more rulebooks are used that include dynamic rules, e.g., rules generated based on traffic conditions, weather conditions, etc. As used herein, a "pre-identified" rule relates to a rule that is identified prior to analysis according to the rule. Such rules can include, for example, known rules for roads, rules relating to collision avoidance strategies, etc. In contrast, a "dynamic" rule can relate to a rule that is based on current circumstances relating to the vehicle or the environment. As described above, such dynamic rules can relate to current traffic conditions, current weather conditions, etc.

[0104] Based on the application of the rules from the rulebook to the paths and corresponding metrics or values, one of the two paths provided to the suggestion comparison system 620 is selected for use by the vehicle. In embodiments, the selection of the path is made according to which of the two paths receives a“best” score based on the comparison of the rules to the paths. Depending on how the metrics or values are generated, the best score can be based on the highest score or the lowest score, where the highest score or the lowest score indicates the path that most adheres to the rules included in the rulebook. In embodiments, the selection of the path can be based on a comparison of the metrics or values to a threshold value (which can be pre-identified or dynamic). For example, in embodiments, the metrics or scores of the individual paths can be analyzed to see if the metrics or scores of the individual paths are greater than (or greater than or equal to) a threshold value. In other embodiments, the analysis can be to see if the metrics or scores of the individual paths are less than (or less than or equal to) a threshold value, depending on how the metrics are calculated. For purposes of discussion, it will be assumed that a metric“passes” if the value of the metric is greater than the threshold value. If only one of the paths has a metric or value that is greater than the threshold value, that path is selected as the path for use by the vehicle. If both of the paths have metrics or values that are greater than the threshold value, both paths are selected. If neither of the paths has a metric or value that is greater than the threshold value, a remedial action can be taken, such as an emergency stop, recalculation of the paths, or some other remedial action. It will be understood that this example is meant to describe only one example embodiment of operation, and that other embodiments can vary.

[0105] The suggestion comparison system 620 then outputs all or part of the selected path or an indication thereof to a time-based trajectory optimization system 625. The time-based trajectory optimization system 625 is configured to calculate a trajectory based on the selected path (e.g., the constrained reference path or the space-based path selected by the suggestion comparison system 620). As used herein, a trajectory refers to information such as path, velocity, or acceleration information. The information related to the trajectory is then provided to the control system 306, which is configured to operate the vehicle according to the path.

[0106] Figure 8 An example technique for path planning according to embodiments is shown. Figure 8 The technique of FIG. 1 is performed, for example, by the planning system 304, and more specifically, by the various subsystems of the planning system 304 as described above with respect to Figure 6 It will be understood that the depicted technique is meant to be one example of such a path planning technique, and that other embodiments can have more or fewer elements than depicted in FIG. 1. In other embodiments, certain elements can occur in a different order than depicted, or simultaneously with each other. Other variations can exist in other embodiments. Figure 8

[0107] ​The techniques include identifying, at 805, a reference path through an environment that includes a subset of a plurality of edges based on a graph that includes the plurality of edges and a plurality of nodes. The graph can include, for example, a graph that is the same as or similar to graph 700. For example, the edges and nodes can include edges and nodes that are the same as or similar to edges 720 and nodes 725. The reference path can include, for example, a reference path that is the same as or similar to reference path 735.

[0108] The techniques also include identifying, at 810, a first path based on an optimization of a spatial model related to the graph and the reference path. The first path can include, for example, a spatial-based path that is the same as or similar to a spatial-based path produced by spatial trajectory optimization system 610 described above.

[0109] The techniques also include identifying, at 815, a second path based on applying at least one constraint to the reference path. The second path can include, for example, a second path that is the same as or similar to a constrained reference path produced by constraint computation system 615.

[0110] The techniques also include selecting, at 820, the first path or the second path as a path to be traversed by an autonomous vehicle based on a pre-identified rulebook. The selection can be the same as or similar to the selection described above, for example, with respect to Figure 6 suggestion comparison system 620. In particular, the selection can be based on a comparison of the paths to the pre-identified rulebook to generate a comparison value or metric. In other embodiments, the comparison can be based on different techniques or algorithms.

[0111] In the preceding description, embodiments of the application have been described with reference to specific details that can vary depending on implementation. Thus, the specification and drawings should be regarded as illustrative rather than restrictive sense. The only and exclusive indication of the scope of the application is the literal and equivalent scope of the claims as issued by the Patent Office, including any subsequent amendments and versions thereof, in the specific form in which it is issued, and the applicant intends that to be the sole basis for determining the scope of the application. Any definitions expressly set forth herein for terms as used in the claims shall govern the meaning of such terms as used in the claims. In addition, when the term "comprising" is used in the preceding description, the following text can be additional steps or entities, or sub-steps / sub-entities of the previously described steps / entities.

Claims

1. A system for a vehicle, wherein, The system includes: At least one processor; and At least one non-transitory computer-related medium, comprising instructions that, when executed by the at least one processor, cause the at least one processor to: Identify reference paths through the environment for a subset of said multiple edges based on a graph that includes multiple edges and multiple nodes; The first path is identified based on the optimization of a spatial model associated with the graph and the reference path, wherein the spatial model is a model used to model the motion of the vehicle in a spatial domain; The second path is identified by applying at least one constraint to the reference path; and The first or second path is selected as the path that the vehicle will traverse based on a pre-identified rulebook. The spatial model optimizes the vehicle state within a prediction range based on at least one free variable and at least one variable related to relaxation in the system. The relaxation represents a constraint that can be violated at a certain cost, and the relaxation includes relaxation on the lateral error, velocity, or acceleration of the vehicle.

2. The system according to claim 1, wherein, The instruction also causes the at least one processor to: Provide the route planning system with an indication of the first route; and The path planning system identifies subsequent reference paths based on at least a portion of the first path.

3. The system according to claim 1, wherein, The instructions also enable the at least one processor to identify the first path based on speed constraints, lane constraints, or obstacles in the environment.

4. The system according to claim 1, wherein, The instructions also enable the at least one processor to identify the second path based on speed constraints, lane constraints, or obstacles in the environment.

5. The system according to claim 1, wherein, The pre-identified rulebook includes rules related to collision prevention or traffic regulations.

6. The system according to claim 1, wherein, The instructions also cause the at least one processor to identify a trajectory based on the selected first path or the second path.

7. The system according to claim 6, wherein, The instructions also cause the at least one processor to output at least one indication of the control to be used by the control system of the vehicle.

8. A method for a vehicle, comprising: The vehicle's at least one processor identifies a reference path through the environment, comprising a subset of the multiple edges, based on a graph comprising multiple edges and multiple nodes; The first path is identified by the at least one processor based on the optimization of a spatial model associated with the graph and the reference path, wherein the spatial model is a model for modeling the motion of the vehicle in a spatial domain; The at least one processor identifies the second path based on applying at least one constraint to the reference path; as well as The at least one processor selects either the first path or the second path as the path that the vehicle will traverse, based on a pre-identified rulebook. The spatial model optimizes the vehicle state within a prediction range based on at least one free variable and at least one variable related to relaxation in the system. The relaxation represents a constraint that can be violated at a certain cost, and the relaxation includes relaxation on the lateral error, velocity, or acceleration of the vehicle.

9. The method according to claim 8, further comprising: The at least one processor identifies subsequent reference paths based on the first path.

10. The method of claim 8, further comprising: The first path or the second path is identified by the at least one processor based on speed constraints, lane constraints, or obstacles in the environment.

11. The method according to claim 8, wherein, The pre-identified rulebook includes rules related to collision prevention or traffic regulations.

12. The method according to claim 8, wherein, The method further includes: the at least one processor identifying a trajectory including speed based on the selected first path or the second path.

13. The method of claim 12, further comprising: The vehicle is controlled by the at least one processor based on the trajectory.

14. One or more non-transitory computer-readable media comprising instructions that, when executed by at least one processor of the vehicle, cause the vehicle to: Identify reference paths through the environment for a subset of said multiple edges based on a graph that includes multiple edges and multiple nodes; The first path is identified based on the optimization of a spatial model associated with the graph and the reference path, wherein the spatial model is a model used to model the motion of the vehicle in a spatial domain; The second path is identified by applying at least one constraint to the reference path; as well as The first or second path is selected based on a pre-identified rulebook as the path that the autonomous vehicle will traverse. The spatial model optimizes the vehicle state within a prediction range based on at least one free variable and at least one variable related to relaxation in the system. The relaxation represents a constraint that can be violated at a certain cost, and the relaxation includes relaxation on the lateral error, velocity, or acceleration of the vehicle.

15. One or more non-transitory computer-readable media according to claim 14, wherein, The instructions are also used to identify subsequent reference paths based on the first path.

16. One or more non-transitory computer-readable media according to claim 14, wherein, The instructions are also used to identify the first path based on speed constraints, lane constraints, or obstacles in the environment.

17. One or more non-transitory computer-readable media according to claim 14, wherein, The instructions are also used to identify the second path based on speed constraints, lane constraints, or obstacles in the environment.

18. One or more non-transitory computer-readable media according to claim 14, wherein, The pre-identified rulebook includes rules related to collision prevention or traffic regulations.

19. One or more non-transitory computer-readable media according to claim 14, wherein, The instructions are also used to identify a trajectory including speed based on the selected first path or second path.

20. One or more non-transitory computer-readable media according to claim 19, wherein, The instructions are also used to instruct the vehicle to traverse the selected first path or the second path according to the identified trajectory.

21. A computer program product comprising a program for causing a computer to perform the method of any one of claims 8-13.

Images