Vehicle, method and storage medium therefor

Abstract

Techniques related to driver data guided spatial planning are described, among other things. A spatial structure comprising a plurality of nodes connected by edges is generated. At least some of the nodes and edges represent a path for navigating a vehicle from a first point to a second point. The edges of the spatial structure are labeled as useful based on a distance metric. The spatial structure is pruned by removing one or more edges from the spatial structure according to their respective labels, where the extent of removal is based on a predetermined graph size, a predetermined performance, or any combination thereof to obtain a pruned graph. A path from the first point to the second point on the pruned graph is identified, and the vehicle is navigated according to the path from the first point to the second point on the pruned graph.

Description

Technical Field

[0001] This manual mainly deals with spatial planning guided by driving data. Background Technology

[0002] Autonomous vehicles offer advantages over human-driven vehicles, including but not limited to reduced road accidents, traffic congestion, parking congestion, and improved fuel efficiency. In making driving decisions, autonomous vehicle systems typically create one or more models of the surrounding environment. Navigation generated by the autonomous vehicle system traverses various solutions to the environment, at least in part, based on said one or more models. Therefore, the model directly impacts the quality of the solutions generated by the autonomous vehicle system. Furthermore, aspects of the model can determine the computational complexity required to generate the solution. Summary of the Invention

[0003] A method for a vehicle includes: generating a spatial structure comprising a plurality of nodes connected by edges using at least one processor, wherein at least a portion of the nodes and edges represent paths for navigating the vehicle from a first point to a second point; marking edges of the spatial structure as useful using at least partially based on a distance metric using the at least one processor; pruning the spatial structure using the at least one processor by removing one or more edges from the spatial structure according to the corresponding markings of the edges, wherein the extent of the removal is based on a predetermined graph size, predetermined performance, or any combination thereof, to obtain a pruned graph; identifying paths from the first point to the second point on the pruned graph using planning circuitry of the vehicle; and navigating the vehicle using control circuitry of the vehicle according to the paths from the first point to the second point on the pruned graph.

[0004] A vehicle includes: at least one computer-readable medium storing computer-executable instructions; and at least one processor configured to execute the computer-executable instructions, the execution of which performs the method.

[0005] A non-transitory computer-readable storage medium includes at least one program for execution by at least one processor of a first device, the at least one program including instructions that, when executed by the at least one processor, cause the first device to perform the method.

[0006] A method for a vehicle includes: evaluating a pruned graph comprising a plurality of nodes connected by edges using at least one processor, wherein the edges of the pruned graph are marked as useful based on a distance metric; iteratively adjusting edges removed from the pruned graph based on corresponding performance of the pruned graph using the at least one processor; selecting the pruned graph with the highest performance using the at least one processor; identifying a path from a first point to a second point on the pruned graph using planning circuitry of the vehicle; and navigating the vehicle according to the path from the first point to the second point on the pruned graph using control circuitry of the vehicle.

[0007] A vehicle includes: at least one computer-readable medium storing computer-executable instructions; and at least one processor configured to execute the computer-executable instructions, the execution of which performs the method.

[0008] A non-transitory computer-readable storage medium includes at least one program executed by at least one processor of a first device, the at least one program including instructions that, when executed by the at least one processor, cause the first device to perform the method. Attached Figure Description

[0009] Figure 1 An example of an autonomous vehicle with autonomous capabilities is shown.

[0010] Figure 2 An example "cloud" computing environment is shown.

[0011] Figure 3 The computer system is shown.

[0012] Figure 4 An example architecture for an autonomous vehicle is shown.

[0013] Figure 5 Examples of inputs and outputs that the perception module can use are shown.

[0014] Figure 6 An example of a LiDAR system is shown.

[0015] Figure 7 The image shows a LiDAR system in operation.

[0016] Figure 8 Additional details on the operation of the LiDAR system are shown.

[0017] Figure 9 A block diagram showing the relationship between the inputs and outputs of the planning module.

[0018] Figure 10 This shows the directed graph used in path planning.

[0019] Figure 11 A block diagram showing the inputs and outputs of the control module is provided.

[0020] Figure 12 A block diagram showing the controller's inputs, outputs, and components is provided.

[0021] Figure 13 This is a diagram of two spatial structures that are suitable for the curvature of a road.

[0022] Figure 14 This is a block diagram of a spatial planning system guided by driving data.

[0023] Figure 15 It is a diagram of the spatial structure.

[0024] Figure 16 This is a flowchart illustrating the processing of spatial planning guided by driving data.

[0025] Figure 17 This is a flowchart illustrating the process of providing feedback during spatial planning guided by driving data. Detailed Implementation

[0026] In the following description, numerous specific details are set forth for purposes of explanation in order to provide a thorough understanding of the invention. However, it will be apparent that the invention may be practiced without these specific details. In other instances, well-known constructions and apparatuses are shown in block diagram form to avoid unnecessarily obscuring the invention.

[0027] In the accompanying drawings, for ease of description, a specific arrangement or order of schematic elements (such as those representing devices, modules, instruction blocks, and data elements) is shown. However, those skilled in the art will understand that the specific order or arrangement of the schematic elements in the drawings is not intended to imply a requirement for a particular processing order or sequence, or a separation of processing procedures. Furthermore, the inclusion of schematic elements in the drawings is not intended to imply that such elements are required in all embodiments, nor is it intended to imply that features represented by such elements cannot be included in some embodiments or cannot be combined with other elements in some embodiments.

[0028] Furthermore, in the accompanying drawings, connecting elements, such as solid or dashed lines or arrows, are used to illustrate connections, relationships, or associations between two or more other schematic elements. The absence of any such connecting element does not imply that connections, relationships, or associations cannot exist. In other words, connections, relationships, or associations between some elements are not shown in the drawings so as not to obscure the content of this disclosure. Additionally, for ease of illustration, a single connecting element is used to represent multiple connections, relationships, or associations between elements. For example, if a connecting element represents communication of signals, data, or instructions, those skilled in the art will understand that such an element represents one or more signal paths (e.g., a bus) that may be necessary to influence the communication.

[0029] Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. Numerous specific details are set forth in the following detailed description in order to provide a thorough understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described can be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0030] The features described below can each be used independently or in any combination with other features. However, any individual feature may not solve any of the problems discussed above, or may only solve one of the problems discussed above. Some of the problems discussed above may not be adequately solved by any of the features described herein. Although headings are provided, information relating to specific headings but not found in the sections bearing those headings can be found elsewhere in this specification. Embodiments are described herein based on the following summary:

[0031] 1. General Overview

[0032] 2. System Overview

[0033] 3. AV Architecture

[0034] 4. AV Input

[0035] 5. AV Planning

[0036] 6. AV Control

[0037] 7. Generation of Spatial Structures

[0038] 8. Driving data-guided spatial planning system

[0039] 9. Determining the useful edges

[0040] General Overview

[0041] Before using graphs to select the routes a vehicle should follow, a simplified graphical representation of the spatial structure can be developed. Typically, spatial planning involves generating a spatial structure that represents multiple locations throughout the space. This spatial structure can be a graph that can be fitted or mapped to road curvature. The path from the origin to the destination is represented by a sequence of edges, where each edge connects multiple nodes in the graph. The edges are labeled, and statistics are derived from at least distance metrics and driver logs. The graph is then pruned based on the labels and statistics.

[0042] Some advantages of these techniques include reducing the size (or density) of the graph based on a predetermined graph size, predetermined performance, or a combination of both. Graph density is directly proportional to the number of available edges in the spatial structure; the denser the graph, the higher the computational complexity for extracting satisfactory plans from a directed graph. As described in this paper, driver-based statistics enable the motion planning module to output higher-quality paths that replicate human-level performance. Furthermore, by reducing the space of possible options and focusing on alternatives that produce better planning performance, the time spent on motion planning is reduced and utilized more efficiently.

[0043] System Overview

[0044] Figure 1 An example of an autonomous vehicle 100 with autonomous capabilities is shown.

[0045] As used herein, the term “autonomy” refers to a function, feature, or facility that enables a vehicle to operate partially or fully without real-time human intervention, including but not limited to fully autonomous vehicles, highly autonomous vehicles, and conditionally autonomous vehicles.

[0046] As used in this article, an autonomous vehicle (AV) is a vehicle with autonomous capabilities.

[0047] As used in this article, "vehicle" includes any mode of transport for goods or people. Examples include cars, buses, trains, airplanes, drones, trucks, ships, vessels, submersibles, and spacecraft. Driverless cars are an example of vehicles.

[0048] As used herein, a “track” refers to a path or route that navigates an AV from a first spatiotemporal location to a second spatiotemporal location. In embodiments, the first spatiotemporal location is referred to as the initial location or starting point, and the second spatiotemporal location is referred to as the destination, final location, target, target location, or target position. In some examples, a track consists of one or more segments (e.g., segments of a road), and each segment consists of one or more blocks (e.g., a lane or part of an intersection). In embodiments, spatiotemporal locations correspond to real-world locations. For example, a spatiotemporal location is a pick-up or drop-off point for people or goods to board or alight.

[0049] As used herein, “(one or more) sensors” includes one or more hardware components for detecting information relating to the environment surrounding the sensor. Some hardware components may include sensing components (e.g., image sensors, biometric sensors), transmission and / or receiving components (e.g., laser or radio frequency wave transmitters and receivers), electronic components (such as analog-to-digital converters), data storage devices (such as RAM and / or non-volatile memory), software or firmware components, and data processing components (such as application-specific integrated circuits), microprocessors, and / or microcontrollers.

[0050] As used herein, a “scene description” is a data structure (e.g., a list) or data stream that includes one or more classified or tagged objects detected by one or more sensors on an AV vehicle, or one or more classified or tagged objects provided by a source outside the AV.

[0051] As used in this article, a "road" is a physical area that can be traversed by vehicles and can correspond to a named passageway (e.g., a city street, an interstate highway, etc.) or an unnamed passageway (e.g., a driveway within a house or office building, a section of a parking lot, a section of an vacant parking lot, a waste disposal area in a rural area, etc.). Because some vehicles (e.g., four-wheel drive pickup trucks, SUVs, etc.) can traverse a variety of physical areas that are not particularly suitable for vehicle travel, a "road" can be any physical area that is not formally defined as a passageway by any municipality or other government or administrative agency.

[0052] As used herein, a "lane" is the portion of a road that can be traversed by vehicles. Sometimes lanes are identified based on lane markings. For example, a lane may correspond to most or all of the space between lane markings, or only a portion of the space between lane markings (e.g., less than 50%). For instance, a road with widely spaced lane markings may accommodate two or more vehicles, allowing one vehicle to overtake another without crossing the lane markings; therefore, this could be interpreted as a lane being narrower than the space between lane markings, or as having two lanes. Lanes can also be interpreted in the absence of lane markings. For example, a lane may be defined based on physical features of the environment (e.g., rocks and trees along a main road in a rural area, or natural obstacles that should be avoided, for example, in underdeveloped areas). Lanes can also be interpreted independently of lane markings or physical features. For example, a lane may be interpreted based on any unobstructed path in an area that would otherwise lack features that would be interpreted as lane boundaries. In the example scenario, an AV could interpret a lane as a lane traversing an unobstructed portion of a field or open space. In another example scenario, an AV can interpret lanes that pass through a wide road (e.g., wide enough for two or more lanes) without lane markings. In this scenario, an AV can communicate lane-related information to other AVs, allowing them to coordinate route planning using the same lane information.

[0053] The term “over-the-air (OTA) client” includes any AV, or any electronic device embedded in, coupled to, or communicating with an AV (e.g., computer, controller, IoT device, electronic control unit (ECU)).

[0054] The term "over-the-air (OTA) update" means any update, alteration, deletion, or addition to software, firmware, data, or configuration settings, or any combination thereof, delivered to an OTA client using proprietary and / or standardized wireless communication technologies, including but not limited to: cellular mobile communications (e.g., 2G, 3G, 4G, 5G), radio local area networks (e.g., WiFi), and / or satellite Internet.

[0055] The term "edge node" refers to one or more edge devices coupled to a network that provide a portal for communicating with AV and can communicate with other edge nodes and cloud-based computing platforms to schedule OTA updates and deliver OTA updates to OTA clients.

[0056] The term "edge device" refers to a device that implements an edge node and provides a physical wireless access point (AP) to the core network of an enterprise or service provider (such as Verizon or AT&T). Examples of edge devices include, but are not limited to: computers, controllers, transmitters, routers, routing switches, integrated access devices (IADs), multiplexers, metropolitan area network (MAN) and wide area network (WAN) access devices.

[0057] "One or more" includes functions performed by a single element, functions performed by multiple elements, such as in a distributed manner, several functions performed by a single element, several functions performed by several elements, or any combination thereof.

[0058] It will also be understood that, although in some cases the terms “first,” “second,” etc., are used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the various described embodiments, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first contact and the second contact are contacts, but they are not the same contact.

[0059] The terminology used in the description of the various embodiments described herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various embodiments described and the appended claims, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that “and / or,” as used herein, refers to and includes any and all possible combinations of one or more of the related list items. It will also be understood that when the terms “comprising,” “including,” “possessing,” and / or “having” are used in this specification, they specifically indicate the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0060] As used herein, depending on the context, the term "if" may optionally be understood as meaning "when" or "at that time" or "in response to being determined" or "in response to being detected." Similarly, depending on the context, the phrase "if determined" or "if [the stated condition or event] has been detected" may optionally be understood as meaning "when determined" or "in response to being determined" or "when [the stated condition or event] is detected" or "in response to being detected."

[0061] As used herein, an AV system refers to an AV and an array of hardware, software, stored data, and real-time generated data that support AV operation. In embodiments, the AV system is incorporated within an AV. In embodiments, the AV system is distributed across several locations. For example, some of the software of the AV system is similar to that described below. Figure 2 The cloud computing environment described is implemented in the cloud computing environment of 200.

[0062] Generally, this document describes technologies applicable to any vehicle with one or more autonomous capabilities, including fully automated vehicle (AV), highly automated vehicle (AV), and conditionally automated vehicle (AV), such as so-called Level 5, Level 4, and Level 3 vehicles, respectively (see SAE International Standard J3016: Classification and Definition of Terms Related to Automated Driving Systems for Motor Vehicles on Roads, the entire contents of which are incorporated herein by reference for further details on vehicle autonomy levels). The technologies described in this document are also applicable to partially automated vehicle (AV) and driver-assisted vehicle (MAV) vehicles, such as so-called Level 2 and Level 1 vehicles (see SAE International Standard J3016: Classification and Definition of Terms Related to Automated Driving Systems for Motor Vehicles on Roads). In embodiments, one or more Level 1, Level 2, Level 3, Level 4, and Level 5 vehicle systems can automatically perform certain vehicle operations (e.g., steering, braking, and map usage) under certain operating conditions based on the processing of sensor inputs. The technologies described in this document can benefit vehicles of any level, ranging from fully automated vehicle (AV) to human-operated vehicles.

[0063] AVs have advantages over vehicles that require human drivers. One advantage is safety. From 1965 to 2015, the number of traffic fatalities per 100 million miles driven in the United States decreased, partly due to additional safety features deployed in vehicles. For example, an extra half-second of warning, presumably related to an impending collision, mitigated many front and rear collisions. However, passive safety features (e.g., seat belts, airbags) may have reached their limits in improving this. Therefore, active safety measures, such as automated vehicle control, are a possible next step in improving these statistics. Since human drivers are considered the cause of serious pre-collision events in most collisions, automated driving systems could potentially achieve better safety outcomes by: more reliably identifying and avoiding emergencies than humans; making better decisions, obeying traffic laws better, and predicting future events better than humans; and controlling the vehicle more reliably than humans.

[0064] refer to Figure 1The AV system 120 enables the vehicle 100 to operate along a trajectory 198, traversing the environment 190 to the destination 199 (sometimes referred to as the final location), while avoiding objects (e.g., natural obstacles 191, vehicles 193, pedestrians 192, cyclists and other obstacles) and complying with road rules (e.g., operating rules or driving preferences).

[0065] In an embodiment, the AV system 120 includes means 101 for receiving and operating operation commands from and on a computer processor 146. The term "operation command" is used to refer to executable instructions (or a set of instructions) that cause a vehicle to perform actions (e.g., driving maneuvers). Operation commands may, without limitation, include instructions for causing the vehicle to begin moving forward, stop moving forward, begin moving backward, stop moving backward, accelerate, decelerate, make a left turn, and make a right turn. In an embodiment, the computer processor 146 is referenced below. Figure 2 The processor 204 described is similar. Examples of the device 101 include a steering controller 102, a brake 103, a gear, an accelerator pedal or other acceleration control mechanism, a windshield wiper, a side door lock, a window controller, and a turn indicator.

[0066] In an embodiment, the AV system 120 includes sensors 121 for measuring or inferring attributes of the state or condition of the vehicle 100, such as the AV's position, linear velocity and angular velocity, linear acceleration and angular acceleration, and heading (e.g., the direction of the front end of the vehicle 100). Examples of sensors 121 are GPS, inertial measurement units (IMUs) that measure both linear acceleration and angular rate of the vehicle, wheel rate sensors for measuring or estimating wheel slip ratio, wheel braking pressure or braking torque sensors, engine torque or wheel torque sensors, and steering angle and angular rate sensors.

[0067] In an embodiment, sensor 121 also includes sensors for sensing or measuring properties of the AV's environment. Examples include a monocular or stereo camera 122 with visible, infrared, or thermal (or both) spectra, a LiDAR 123, a RADAR, an ultrasonic sensor, a time-of-flight (TOF) depth sensor, a rate sensor, a temperature sensor, a humidity sensor, and a precipitation sensor.

[0068] In one embodiment, the AV system 120 includes a data storage unit 142 and a memory 144 for storing machine instructions associated with a computer processor 146 or data collected by the sensor 121. In another embodiment, the data storage unit 142 is associated with the following... Figure 2The described ROM 208 or storage device 210 is similar. In this embodiment, memory 144 is similar to main memory 206 described below. In this embodiment, data storage unit 142 and memory 144 store historical, real-time, and / or predictive information about environment 190. In this embodiment, the stored information includes maps, driving performance, traffic congestion updates, or weather conditions. In this embodiment, data related to environment 190 is transmitted from remote database 134 to vehicle 100 via a communication channel.

[0069] In an embodiment, the AV system 120 includes communication devices 140 for transmitting measured or inferred attributes of the state and conditions of other vehicles, such as position, linear velocity and angular velocity, linear acceleration and angular acceleration, and linear heading and angular heading, to vehicle 100. These devices include vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication devices, as well as devices for wireless communication via point-to-point or ad hoc networks, or both. In an embodiment, communication device 140 communicates across the electromagnetic spectrum (including radio and optical communications) or other media (e.g., air and acoustic media). The combination of vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) communication (and in some embodiments, one or more other types of communication) is sometimes referred to as vehicle-to-all-things (V2X) communication. V2X communication typically conforms to one or more communication standards for communication with and between autonomous vehicles.

[0070] In one embodiment, the communication device 140 includes a communication interface. For example, a wired, wireless, WiMAX, Wi-Fi, Bluetooth, satellite, cellular, optical, near-field, infrared, or radio interface. The communication interface transmits data from a remote database 134 to the AV system 120. In one embodiment, the remote database 134 is embedded in a cloud computing environment 200. The communication device 140 transmits data collected from the sensor 121 or other data related to the operation of the vehicle 100 to the remote database 134. In one embodiment, the communication device 140 transmits information related to teleoperation to the vehicle 100. In some embodiments, the vehicle 100 communicates with other remote (e.g., "cloud") servers 136.

[0071] In this embodiment, the remote database 134 also stores and transmits digital data (e.g., data such as road and street locations). This data is stored in memory 144 on the vehicle 100 or transmitted from the remote database 134 to the vehicle 100 via a communication channel.

[0072] In one embodiment, the remote database 134 stores and transmits historical information (e.g., rate and acceleration distribution) related to driving attributes of vehicles that previously traveled along trajectory 198 at similar times of day. In one implementation, such data may be stored in memory 144 on vehicle 100 or transmitted from the remote database 134 to vehicle 100 via a communication channel.

[0073] The computer processor 146 located on the vehicle 100 generates control actions in an algorithmic manner based on both real-time sensor data and prior information, allowing the AV system 120 to perform its autonomous driving capabilities.

[0074] In one embodiment, the AV system 120 includes a computer peripheral device 132 coupled to a computer processor 146 for providing information and alerts to a user of the vehicle 100 (e.g., a passenger or a remote user) and receiving input from that user. In another embodiment, the peripheral device 132 is similar to the one described in the following reference. Figure 2 The discussed display 212, input device 214, and cursor controller 216 are coupled wirelessly or wiredly. Any two or more interface devices can be integrated into a single device.

[0075] In one embodiment, the AV system 120 receives and enforces a privacy level for an occupant, such as one specified by the occupant or stored in a profile associated with the occupant. The occupant's privacy level determines how access is permitted to specific occupant-related information (e.g., occupant comfort data, biometric data, etc.) stored in the occupant profile and / or stored on cloud server 136 and associated with the occupant profile. In one embodiment, the privacy level specifies specific occupant-related information that is deleted once the ride is complete. In another embodiment, the privacy level specifies specific occupant-related information and identifies one or more entities authorized to access that information. Examples of the specified entities authorized to access the information may include other AV systems, third-party AV systems, or any entity that could potentially access the information.

[0076] An occupant's privacy level can be specified at one or more granular levels. In one embodiment, the privacy level identifies specific information to be stored or shared. In another embodiment, the privacy level applies to all information associated with the occupant, allowing the occupant to specify that her personal information should not be stored or shared. The designation of entities authorized to access specific information can also be specified at various granular levels. The various sets of entities authorized to access specific information may include, for example, other AVs, cloud server 136, specific third-party AV systems, etc.

[0077] In an embodiment, AV system 120 or cloud server 136 determines whether AV 100 or another entity can access certain information associated with an occupant. For example, a third-party AV system attempting to access occupant input related to a specific time and place must, for example, obtain authorization from AV system 120 or cloud server 136 to access occupant-related information. For example, AV system 120 uses a specified privacy level for the occupant to determine whether location- and time-related occupant input can be presented to a third-party AV system, AV 100, or another AV. This allows the occupant's privacy level to specify which other entities are allowed to receive data related to the occupant's actions or other data associated with the occupant.

[0078] Figure 2 This illustrates an example "cloud" computing environment. Cloud computing is a service delivery model that enables convenient, on-demand access over a network to a shared pool of configurable computing resources, such as networks, network bandwidth, servers, processing power, memory, storage, applications, virtual machines, and services. In a typical cloud computing system, one or more large cloud data centers house the machines used to deliver the services provided by the cloud. Now refer to... Figure 2 The cloud computing environment 200 includes cloud data centers 204a, 204b, and 204c interconnected via cloud 202. Data centers 204a, 204b, and 204c provide cloud computing services to computer systems 206a, 206b, 206c, 206d, 206e, and 206f connected to cloud 202.

[0079] A cloud computing environment 200 includes one or more cloud data centers. Generally, a cloud data center (e.g.) Figure 2 The cloud data center 204a shown refers to the cloud (e.g., Figure 2 The physical arrangement of servers in cloud 202 (or a specific portion of the cloud) is illustrated. For example, servers are physically arranged in rooms, groups, rows, and racks within a cloud data center. A cloud data center has one or more regions, each containing one or more server rooms. Each room has one or more rows of servers, and each row includes one or more racks. Each rack includes one or more individual server nodes. In some implementations, servers in regions, rooms, racks, and / or rows are arranged into groups based on the physical infrastructure requirements of the data center facility, including power, energy, heat, heat sources, and / or other requirements. In this embodiment, server nodes are similar to... Figure 3 The computer system described herein. Data center 204a has many computing systems distributed across multiple racks.

[0080] Cloud 202 includes cloud data centers 204a, 204b, and 204c, and networks and network resources (e.g., network devices, nodes, routers, switches, and network cables) for connecting cloud data centers 204a, 204b, and 204c and facilitating access to cloud computing services by computing systems 206a-f. In embodiments, the network represents one or more local area networks, wide area networks, or any combination of wired or wireless networks coupled using terrestrial or satellite connections. Data exchanged over the network is transmitted using various network layer protocols, such as Internet Protocol (IP), Multiprotocol Label Switching (MPLS), Asynchronous Transfer Mode (ATM), Frame Relay, etc. Furthermore, in embodiments where the network represents a combination of multiple subnetworks, different network layer protocols are used on each underlying subnetwork. In some embodiments, the network represents one or more interconnected internetworks (such as the public Internet).

[0081] Computing systems 206a-f or cloud computing service consumers connect to the cloud 202 via network links and network adapters. In embodiments, computing systems 206a-f are implemented as various computing devices, such as servers, desktops, laptops, tablets, smartphones, Internet of Things (IoT) devices, AVs (including cars, drones, space shuttles, trains, buses, etc.), and consumer electronics. In embodiments, computing systems 206a-f are implemented in other systems or as part of other systems.

[0082] Figure 3 A computer system 300 is illustrated. In an implementation, the computer system 300 is a dedicated computing device. The dedicated computing device is hardwired to perform these technologies, or includes a digital electronic device such as one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs) that is persistently programmed to perform the aforementioned technologies, or may include one or more general-purpose hardware processors programmed to perform these technologies according to program instructions in firmware, memory, other memory, or a combination thereof. Such a dedicated computing device may also combine custom hardwired logic, ASICs, or FPGAs with custom programming to accomplish these technologies. In various embodiments, the dedicated computing device is a desktop computer system, a portable computer system, a handheld device, a network device, or any other device that includes hardwired and / or program logic to implement these technologies.

[0083] In one embodiment, the computer system 300 includes a bus 302 or other communication mechanism for conveying information, and a processor 304 coupled to the bus 302 to process information. The processor 304 is, for example, a general-purpose microprocessor. The computer system 300 also includes a main memory 306, such as random access memory (RAM) or other dynamic storage device, coupled to the bus 302 to store information and instructions executed by the processor 304. In one implementation, the main memory 306 is used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 304. When these instructions are stored in a non-transitory storage medium accessible to the processor 304, the computer system 300 becomes a dedicated machine customized to perform the operations specified in the instructions.

[0084] In an embodiment, the computer system 300 further includes a read-only memory (ROM) 308 or other static storage device coupled to the bus 302 for storing static information and instructions of the processor 304. A storage device 310, such as a disk, optical disk, solid-state drive, or three-dimensional cross-point memory, is provided and coupled to the bus 302 to store information and instructions.

[0085] In this embodiment, the computer system 300 is coupled via a bus 302 to a display 312, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light-emitting diode (LED) display, or an organic light-emitting diode (OLED) display for displaying information to a computer user. An input device 314, including alphanumeric keys and other keys, is coupled to the bus 302 for transmitting information and command selections to the processor 304. Another type of user input device is a cursor controller 316, such as a mouse, trackball, touchscreen, or cursor arrow keys, for transmitting directional information and command selections to the processor 304 and for controlling the movement of the cursor on the display 312. Such input devices typically have two degrees of freedom on two axes (a first axis (e.g., the x-axis) and a second axis (e.g., the y-axis)), which allow the device to specify a position in a plane.

[0086] According to one embodiment, the techniques described herein are executed by computer system 300 in response to processor 304 executing one or more sequences of one or more instructions contained in main memory 306. These instructions are read into main memory 306 from another storage medium, such as storage device 310. Executing the sequence of instructions contained in main memory 306 causes processor 304 to perform the process steps described herein. In alternative embodiments, hardwired circuitry is used instead of or in combination with software instructions.

[0087] As used herein, the term "storage medium" refers to any non-transitory medium that stores data and / or instructions that enable a machine to operate in a particular manner. Such storage media include non-volatile media and / or volatile media. Non-volatile media include, for example, optical discs, magnetic disks, solid-state drives, or three-dimensional cross-point memory such as storage device 310. Volatile media include dynamic memory, such as main memory 306. Common forms of storage media include, for example, floppy disks, floppy disks, hard disks, solid-state drives, magnetic tape or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with perforations, RAM, PROMs and EPROMs, FLASH-EPROMs, NV-RAMs, or any other memory chips or memory cartridges.

[0088] Storage media differ from transmission media, but can be used in conjunction with them. Transmission media participate in the information transmission between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, which include wires with a bus 302. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communication.

[0089] In embodiments, various forms of media involve carrying one or more sequences of one or more instructions to processor 304 for execution. For example, these instructions may initially be executed on a disk or solid-state drive of a remote computer. The remote computer loads the instructions into its dynamic memory and transmits them over a telephone line using a modem. A local modem of computer system 300 receives data over the telephone line and converts the data into an infrared signal using an infrared transmitter. An infrared detector receives the data carried in the infrared signal, and appropriate circuitry places the data on bus 302. Bus 302 carries the data to main memory 306, from which processor 304 retrieves and executes the instructions. The instructions received by main memory 306 may optionally be stored on storage device 310 before or after execution by processor 304.

[0090] Computer system 300 also includes a communication interface 318 coupled to bus 302. Communication interface 318 provides bidirectional data communication coupled to network link 320 connected to local network 322. For example, communication interface 318 is an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem used to provide data communication connectivity with a corresponding type of telephone line. As another example, communication interface 318 is a Local Area Network (LAN) card used to provide data communication connectivity with a compatible LAN. In some implementations, a wireless link is also implemented. In any such implementation, communication interface 318 transmits and receives electrical, electromagnetic, or optical signals carrying digital data streams representing various types of information.

[0091] Network link 320 typically provides data communication to other data devices via one or more networks. For example, network link 320 provides connectivity to host computer 324 or to a cloud data center or device operated by Internet Service Provider (ISP) 326 via local network 322. ISP 326, in turn, provides data communication services via a worldwide packet data communication network now commonly referred to as the "Internet" 328. Both local network 322 and Internet 328 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks and signals on network link 320 via communication interface 318 are example forms of transmission media carrying digital data entering and leaving computer system 300. In embodiments, network 320 includes the aforementioned cloud 202 or a portion of cloud 202.

[0092] Computer system 300 sends messages and receives data including program code through one or more networks, network links 320, and communication interfaces 318. In an embodiment, computer system 300 receives code for processing. The received code is executed by processor 304 upon receipt and / or stored in storage device 310, or in other non-volatile storage devices for later execution.

[0093] AV architecture

[0094] Figure 4 Showing for AV (e.g., Figure 1 The example architecture 400 of the vehicle 100 shown is illustrated. Architecture 400 includes a sensing module 402 (sometimes called a sensing circuit), a planning module 404 (sometimes called a planning circuit), a control module 406 (sometimes called a control circuit), a positioning module 408 (sometimes called a positioning circuit), and a database module 410 (sometimes called a database circuit). Each module plays a role in the operation of the vehicle 100. Commonly, modules 402, 404, 406, 408, and 410 can be... Figure 1 This is part of the AV system 120 shown. In some embodiments, any of modules 402, 404, 406, 408, and 410 is a combination of computer software (e.g., executable code stored on a computer-readable medium) and computer hardware (e.g., one or more microprocessors, microcontrollers, application-specific integrated circuits (ASICs), hardware memory devices, other types of integrated circuits, other types of computer hardware, or any or all combinations of these hardware). Modules 402, 404, 406, 408, and 410 are each sometimes referred to as processing circuitry (e.g., computer hardware, computer software, or a combination of both). Any or all combinations of modules 402, 404, 406, 408, and 410 are also examples of processing circuitry.

[0095] In use, the planning module 404 receives data representing the destination 412 and determines data representing the trajectory 414 (sometimes called a route) that the vehicle 100 can travel to reach (e.g., arrive at) the destination 412. In order for the planning module 404 to determine the data representing the trajectory 414, the planning module 404 receives data from the sensing module 402, the positioning module 408, and the database module 410.

[0096] The sensing module 402 is used, for example, as follows Figure 1 One or more sensors 121 are shown to identify nearby physical objects. The objects are classified (e.g., grouped into types such as pedestrians, bicycles, cars, traffic signs, etc.), and a scene description including the classified objects 416 is provided to the planning module 404.

[0097] The planning module 404 also receives data representing the location 418 of the AV from the positioning module 408. The positioning module 408 determines the location of the AV by using data from sensor 121 and data (e.g., geographic data) from database module 410. For example, the positioning module 408 uses data from GNSS (Global Navigation Satellite System) sensors and geographic data to calculate the longitude and latitude of the AV. In embodiments, the data used by the positioning module 408 includes high-precision maps with lane geometry properties, maps describing road network connectivity properties, maps describing lane physical properties (such as traffic speed, traffic volume, number of vehicle and bicycle lanes, lane width, lane traffic direction, or lane marking type and location, or combinations thereof), and maps describing the spatial locations of road features (such as intersections, traffic signs, or various types of other traffic signals). In embodiments, the high-precision map is constructed by adding data to a low-precision map via automatic or manual annotation.

[0098] The control module 406 receives data representing trajectory 414 and data representing AV position 418, and operates the AV control functions 420a-420c (e.g., steering, throttle, braking, ignition) in a manner that will cause the vehicle 100 to travel along trajectory 414 to reach destination 412. For example, if trajectory 414 includes a left turn, the control module 406 will operate the control functions 420a-420c in such a way that the steering angle of the steering function will cause the vehicle 100 to turn left, and the throttle and brake will cause the vehicle 100 to pause before turning and wait for passing pedestrians or vehicles.

[0099] AV input

[0100] Figure 5 The sensing module 402 is shown. Figure 4The inputs used are 502a-502d (e.g., Figure 1 Examples of sensor 121 and outputs 504a-504d (e.g., sensor data) are shown. One input 502a is a LiDAR (light detection and ranging) system (e.g., Figure 1 The LiDAR system shown is 123. LiDAR is a technique that uses light (e.g., a beam of light such as infrared light) to obtain data related to physical objects in its line of sight. The LiDAR system produces LiDAR data as output 504a. For example, LiDAR data is a collection of 3D or 2D points (also called point clouds) used to construct a representation of environment 190.

[0101] Another input 502b is a RADAR (radar) system. RADAR is a technology that uses radio waves to obtain data related to nearby physical objects. RADAR can obtain data related to objects that are not within the line of sight of a LiDAR system. The RADAR system generates RADAR data as output 504b. For example, RADAR data is one or more radio frequency electromagnetic signals used to construct a representation of environment 190.

[0102] Another input 502c is a camera system. The camera system uses one or more cameras (e.g., a digital camera using a light sensor such as a charge-coupled device [CCD]) to acquire information about nearby physical objects. The camera system produces camera data as output 504c. Camera data is typically in the form of image data (e.g., data in image data formats such as RAW, JPEG, PNG, etc.). In some examples, the camera system has multiple independent cameras, for example, for stereoscopic imaging (stereoscopic vision), which enables the camera system to perceive depth. Although the objects perceived by the camera system are described here as "nearby," this is relative to the AV (view of objects). In some embodiments, the camera system is configured to "see" distant objects (e.g., objects as far as 1 kilometer or more in front of the AV). Therefore, in some embodiments, the camera system has features such as sensors and lenses optimized for perceiving distant objects.

[0103] Another input 502d is a Traffic Light Detection (TLD) system. The TLD system uses one or more cameras to acquire information related to traffic lights, street signs, and other physical objects that provide visual navigation information. The TLD system produces TLD data as output 504d. TLD data is often in the form of image data (e.g., data in image data formats such as RAW, JPEG, PNG, etc.). The TLD system differs from systems that include cameras in that it uses cameras with a wide field of view (e.g., using a wide-angle lens or fisheye lens) to acquire information related to as many physical objects as possible that provide visual navigation information, enabling the vehicle 100 to access all relevant navigation information provided by these objects. For example, the TLD system has a field of view of approximately 120 degrees or greater.

[0104] In some embodiments, sensor fusion technology is used to combine outputs 504a-504d. Thus, individual outputs 504a-504d are provided to other systems of the vehicle 100 (e.g., to systems such as...). Figure 4 The planning module 404 shown may provide combined outputs to other systems in the form of single or multiple combined outputs of the same type (e.g., using the same combination technique or combining the same outputs or both) or single or multiple combined outputs of different types (e.g., using different individual combination techniques or combining different individual outputs or both). In some embodiments, early fusion techniques are used. Early fusion techniques are characterized by combining the outputs before applying one or more data processing steps to the combined outputs. In some embodiments, late fusion techniques are used. Late fusion techniques are characterized by combining the outputs after applying one or more data processing steps to the individual outputs.

[0105] Figure 6 An example of a LiDAR system 602 is shown (e.g., Figure 5The input 502a is shown. The LiDAR system 602 emits light 604a-604c from a emitter 606 (e.g., a laser emitter). The light emitted by the LiDAR system is typically not in the visible spectrum; for example, infrared light is often used. Some of the emitted light 604b encounters a physical object 608 (e.g., a vehicle) and is reflected back to the LiDAR system 602. (The light emitted from the LiDAR system typically does not penetrate the physical object, e.g., a solid physical object.) The LiDAR system 602 also has one or more photodetectors 610 for detecting the reflected light. In an embodiment, one or more data processing systems associated with the LiDAR system generate an image 612 representing the field of view 614 of the LiDAR system. Image 612 includes information representing the boundary 616 of the physical object 608. Thus, image 612 is used to determine the boundary 616 of one or more physical objects near the AV.

[0106] Figure 7 The diagram illustrates a LiDAR system 602 in operation. In the scenario shown, the vehicle 100 receives both a camera system output 504c in the form of an image 702 and a LiDAR system output 504a in the form of LiDAR data points 704. In use, the vehicle 100's data processing system compares the image 702 with the data points 704. Specifically, physical objects 706 identified in the image 702 are also identified in the data points 704. Thus, the vehicle 100 perceives the boundaries of physical objects based on the contours and density of the data points 704.

[0107] Figure 8 Additional details of the operation of the LiDAR system 602 are shown. As described above, the vehicle 100 detects the boundaries of physical objects based on the characteristics of the data points detected by the LiDAR system 602. Figure 8 As shown, a flat object, such as ground 802, will reflect light 804a-804d emitted from LiDAR system 602 in a consistent manner. In other words, because LiDAR system 602 emits light at a consistent interval, ground 802 will reflect light back to LiDAR system 602 at the same consistent interval. When vehicle 100 travels on ground 802, LiDAR system 602 will continue to detect light reflected by the next effective surface point 806 if nothing obstructs its path. However, if object 808 obstructs its path, the light 804e-804f emitted by LiDAR system 602 will be reflected from points 810a-810b in a manner inconsistent with the expected consistency. Based on this information, vehicle 100 can determine the presence of object 808.

[0108] AV planning

[0109] Figure 9 Show (for example, as) Figure 4 The diagram 900 illustrates the relationship between the inputs and outputs of the planning module 404. Generally, the output of the planning module 404 is a route 902 from a starting point 904 (e.g., a source location or initial location) to an ending point 906 (e.g., a destination or final location). Route 902 is typically defined by one or more road segments. For example, a road segment refers to the distance to be traveled over at least a portion of a street, road, highway, driveway, or other physical area suitable for vehicle travel. In some examples, such as if the vehicle 100 is an off-road capable vehicle such as a four-wheel drive (4WD) or all-wheel drive (AWD) car, SUV, or pickup truck, route 902 includes “off-road” segments such as unpaved paths or open fields.

[0110] In addition to route 902, the planning module also outputs lane-level route planning data 908. Lane-level route planning data 908 is used to navigate segments of route 902 at specific times based on conditions. For example, if route 902 comprises a multi-lane highway, lane-level route planning data 908 includes trajectory planning data 910, which vehicle 100 can use to select a lane from the multiple lanes based on factors such as whether an exit is nearby, whether other vehicles are present in one or more lanes, or other factors that change over a period of minutes or less. Similarly, in some implementations, lane-level route planning data 908 includes a speed constraint 912 specific to a segment of route 902. For example, if the segment includes pedestrians or unexpected traffic, speed constraint 912 can limit AV 100 to a slower speed than expected, such as based on speed limit data for that segment.

[0111] In this embodiment, the input to the planning module 404 includes (e.g., from...) Figure 4 The database module 410 shown contains database data 914 and current location data 916 (for example, Figure 4 The AV position shown is 418), (for example, for use with Figure 4 The destination data 918 and object data 920 shown for destination 412 (e.g., as shown) Figure 4The perception module 402 shown perceives classified objects 416. In some embodiments, database data 914 includes rules used during planning. The rules are specified using a formal language (e.g., Boolean logic). At least some of these rules will apply to any given situation encountered by vehicle 100. A rule applies to a given situation if it has conditions satisfied based on information available to vehicle 100 (e.g., information about the surrounding environment). Rules can have priorities. For example, a rule "move to the leftmost lane if the road is a highway" can have a lower priority than "move to the rightmost lane if the exit is within a mile."

[0112] Figure 10 This is illustrated in path planning (e.g., by planning module 404). Figure 4 The directed graph used is 1000. Generally speaking, such as... Figure 10 The directed graph 1000 shown is used to determine any path between a starting point 1002 and an ending point 1004. In the real world, the distance separating the starting point 1002 and the ending point 1004 may be relatively large (e.g., in two different urban areas) or relatively small (e.g., two intersections adjacent to a city block or two lanes of a multi-lane road).

[0113] In an embodiment, the directed graph 1000 has nodes 1006a-1006d representing different locations that a vehicle 100 may occupy between a starting point 1002 and an ending point 1004. In some examples, for instance, when the starting point 1002 and the ending point 1004 represent different urban areas, nodes 1006a-1006d represent road segments. In some examples, for instance, when the starting point 1002 and the ending point 1004 represent different locations on the same road, nodes 1006a-1006d represent different locations on that road. Thus, the directed graph 1000 includes information at different levels of granularity. In an embodiment, the directed graph with higher granularity is also a subgraph of another directed graph with a larger scale. For example, most of the information in a directed graph where the starting point 1002 and the ending point 1004 are far apart (e.g., many miles apart) is at a low granularity, and the directed graph is based on stored data, but the directed graph also includes some high-granularity information for representing a portion of the physical location in the field of view of the vehicle 100.

[0114] Nodes 1006a-1006d are distinct from objects 1008a-1008b that cannot overlap with nodes. In an embodiment, at a low granularity, objects 1008a-1008b represent areas that vehicles cannot pass through, such as areas without streets or roads. At a high granularity, objects 1008a-1008b represent physical objects within the field of view of vehicle 100, such as other vehicles, pedestrians, or other entities with which vehicle 100 cannot share physical space. In an embodiment, some or all of objects 1008a-1008b are static objects (e.g., objects that do not change position, such as streetlights or utility poles) or dynamic objects (e.g., objects that can change position, such as pedestrians or other cars).

[0115] Nodes 1006a-1006d are connected by edges 1010a-1010c. If two nodes 1006a-1006b are connected by edge 1010a, then vehicle 100 can travel between one node 1006a and the other node 1006b, for example, without having to travel to an intermediate node before reaching the other node 1006b. (When it is mentioned that vehicle 100 travels between nodes, it means that vehicle 100 travels between two physical locations represented by the respective nodes.) Edges 1010a-1010c are typically bidirectional, meaning that vehicle 100 can travel from a first node to a second node, or from a second node to a first node. In an embodiment, edges 1010a-1010c are unidirectional, meaning that vehicle 100 can travel from a first node to a second node, but not from a second node to a first node. When edges 1010a-1010c represent, for example, a one-way street, a single lane of a street, road, or highway, or other features that can only be traversed in one direction due to legal or physical constraints, edges 1010a-1010c are one-way.

[0116] In an embodiment, the planning module 404 uses a directed graph 1000 to identify a path 1012 consisting of nodes and edges between the start point 1002 and the end point 1004.

[0117] Edges 1010a-1010c have associated costs 1014a-1014b. Costs 1014a-1014b represent the resources that would be spent if vehicle 100 selected that edge. A typical resource is time. For example, if the physical distance represented by one edge 1010a is twice the physical distance represented by another edge 1010b, then the associated cost 1014a of the first edge 1010a can be twice the associated cost 1014b of the second edge 1010b. Other factors affecting time include anticipated traffic, the number of intersections, speed limits, etc. Another typical resource is fuel economy. The two edges 1010a-1010b can represent the same physical distance, but due to factors such as road conditions and anticipated weather, one edge 1010a may require more fuel than the other edge 1010b.

[0118] When the planning module 404 identifies the path 1012 between the starting point 1002 and the ending point 1004, the planning module 404 typically selects the path that is optimized for cost, such as the path that has the minimum total cost when the individual costs of the edges are added together.

[0119] AV control

[0120] Figure 11 Show (for example, as) Figure 4 The block diagram 1100 shows the inputs and outputs of the control module 406. The control module operates according to a controller 1102, which includes, for example, one or more processors similar to processor 304 (e.g., one or more computer processors such as a microprocessor or microcontroller or both); short-term and / or long-term data storage devices similar to main memory 306, ROM 308 and storage device 310 (e.g., memory, random access memory or flash memory or both); and instructions stored in the memory that, when executed (e.g. by one or more processors), perform the operation of controller 1102.

[0121] In one embodiment, controller 1102 receives data representing a desired output 1104. The desired output 1104 typically includes speed, such as rate and heading. The desired output 1104 may be based, for example, from (e.g., as...) Figure 4The planning module 404 receives the data shown. Based on the desired output 1104, the controller 1102 generates data that can be used as throttle input 1106 and steering input 1108. Throttle input 1106 indicates the magnitude of the desired output 1104 by engaging the throttle of the vehicle 100 (e.g., acceleration control), for example, by engaging the steering pedal or engaging another throttle control. In some examples, throttle input 1106 also includes data that can be used to engage the brakes of the vehicle 100 (e.g., deceleration control). Steering input 1108 indicates the steering angle, such as the steering control of the AV (e.g., steering wheel, steering angle actuator, or other function for controlling the steering angle), which should be positioned to achieve the desired output 1104.

[0122] In one embodiment, controller 1102 receives feedback used when adjusting inputs provided to throttle and steering. For example, if vehicle 100 encounters an obstacle 1110 such as a hill, the measured rate 1112 of vehicle 100 drops below the desired output rate. In another embodiment, any measured output 1114 is provided to controller 1102 so that necessary adjustments can be made, for example, based on the difference 1113 between the measured rate and the desired output. The measured output 1114 includes measured position 1116, measured speed 1118 (including rate and heading), measured acceleration 1120, and other sensor-measurable outputs of vehicle 100.

[0123] In one embodiment, information related to interference 1110 is detected in advance, for example, by a sensor such as a camera or LiDAR sensor, and this information is provided to the predictive feedback module 1122. The predictive feedback module 1122 then provides information that the controller 1102 can use to make appropriate adjustments. For example, if the vehicle 100's sensors detect ("see") a hill, the controller 1102 can use this information to prepare to engage the throttle at an appropriate time to avoid significant deceleration.

[0124] Figure 12 A block diagram 1200 shows the inputs, outputs, and components of controller 1102. Controller 1102 has a rate analyzer 1202 that influences the operation of throttle / brake controller 1204. For example, the rate analyzer 1202 instructs throttle / brake controller 1204 to accelerate or decelerate using throttle / brake 1206 based on feedback received by, for example, controller 1102 and processed by the rate analyzer 1202.

[0125] The controller 1102 also has a lateral tracking controller 1208 that affects the operation of the steering wheel controller 1210. For example, the lateral tracking controller 1208 instructs the steering wheel controller 1210 to adjust the position of the steering angle actuator 1212 based on feedback received by the controller 1102 and processed by the lateral tracking controller 1208.

[0126] Controller 1102 receives several inputs for determining how to control the throttle / brake 1206 and the steering angle actuator 1212. Planning module 404 provides controller 1102 with information such as selecting the heading of vehicle 100 at the start of operation and determining which road segment vehicle 100 will cross when it reaches an intersection. Positioning module 408 provides controller 1102 with information describing the current location of vehicle 100, such as allowing controller 1102 to determine whether vehicle 100 is at the expected location based on the positive control of the throttle / brake 1206 and steering angle actuator 1212. In this embodiment, controller 1102 receives information from other inputs 1214, such as information received from a database, computer network, etc.

[0127] Generation of spatial structures

[0128] Figure 13 This is a diagram illustrating two spatial structures suitable for the curvature of a road. Figure 13 In the example, the spatial structure is a spatial diagram. A spatial diagram can be, for example, about... Figure 10 The directed graph described is a directed graph 1000, etc. Spatial graph 1302 is shown along road 1304A, while spatial graph 1312 is shown along road 1304B. Spatial graphs 1302 and 1312 respectively include multiple edges 1306 and 1316 between the start and end points (e.g., Figure 10 Edges 1010a-1010c). The start and end points are nodes of the spatial graph (e.g., Figure 10 Nodes 1006a-1006d). Planning module (e.g., Figure 4 The planning module 404 receives a spatial map as input and determines a representation of the vehicle (e.g., Figure 1 The vehicle (100) can navigate the trajectory data. Therefore, the path from the starting point to the ending point is represented by a series of edges, where each edge connects multiple nodes of the graph. In one embodiment, an edge represents a series of adjacent locations in space forming a line between a first point and a second point in the spatial graph. In another embodiment, an edge represents an area where a strip or line of width is formed between the first point and the second point in the spatial graph. For example, a wide edge between the first and second points may cover the area around the centerline of a lane. In some cases, the size of the strip or the width of the line is a predetermined value.

[0129] During route planning, the planning module receives information including the destination (e.g., Figure 9 The spatial structure of the destination data (918), and determine the trajectory (e.g., the path that a vehicle can travel to reach (e.g., arrive at) the destination). Figure 9 The trajectory planning data (910) or route data. In other words, the planning module identifies the path that the vehicle can traverse between the origin and the destination. The planning module bases its decisions on one or more parameters (such as time, cost, etc.). Figure 10 The cost (1014a, 1014c), traffic flow, number of intersections, speed limits, temporary obstacles, or any combination thereof are used to optimize the path. The spatial structure used for planning has a direct impact on the performance of the planning module. Generally, dense graphs produce trajectories that more closely approximate human driving behavior compared to trajectories derived from less dense graphs. However, extracting the final trajectory from a dense graph requires a higher level of computational complexity and additional work from the planning module. The higher computational complexity results in a greater number of possible trajectories available when using dense graphs. Furthermore, dense graphs typically do not produce optimal planning solutions in real time.

[0130] Compared to Figure 1302, Figure 1312 has been pruned or simplified to include edges identified as relevant to the vehicle or otherwise useful. Simplifying the edges in the graph to those identified as relevant, useful, or otherwise important for path planning also simplifies subsequent calculations performed by the planning module. The trajectory derived from the simplified graph, including edges marked as useful, exhibits the same high quality as the trajectory derived from the dense graph.

[0131] Typically, an edge is marked as useful if it meets or exceeds a threshold applied to edges in a spatial graph. In this example, the threshold is based on the cost associated with the edge (e.g., ...). Figure 10 The cost (1014a, 1014c), the number of times the edge is found in the human driving data, or any combination thereof. Otherwise, the edge is marked as useless. In an embodiment, edges are marked as useful based on road geometry (including lane centerlines). For example, edges falling below a predetermined maximum distance from the lane centerline are marked as useful, while edges above the predetermined maximum distance from the lane centerline are marked as useless.

[0132] In this embodiment, edges are labeled as useful or useless by comparing them to actual human driving data, and based at least in part on a distance metric. Human driving data represents the actual ground conditions or desired outcomes when a vehicle traverses a physical area. Human driving data may be stored in a driving log. In this embodiment, machine learning techniques are used to label edges as useful or useless. For example, a classification module is performed to classify each edge as useful or useless. Classifying each edge as useful or useless is associated with a confidence score that indicates the probability of a correct classification.

[0133] Figure 13 The figures are not intended to indicate that figures 1302 and 1312 will include Figure 13 All components shown. Instead, Figures 1302 and 1312 may include fewer components or Figure 13 Additional components not shown (e.g., additional roads, edges, nodes, etc.). Depending on the specific implementation details, Figures 1302 and 1312 may include any number of additional components not shown. Furthermore, any graph generation can be implemented partially or entirely in hardware and / or a processor. For example, this function can be implemented using an application-specific integrated circuit, in logic implemented in a processor, in logic implemented in a dedicated graph processing unit, or in any other device.

[0134] Driving data-guided spatial planning system

[0135] Figure 14 This is a block diagram of a driving data-guided spatial planning system 1400. The spatial planning system 1400 is used to generate a simplified spatial structure including useful edges. A graph layout generator 1402 generates a spatial graph layout that well covers the space without concern for the size or density of the spatial structure. Generally, coverage refers to the ability to represent all possible paths throughout the space. The graph output by the graph layout generator 1402 can be referred to as the original graph and can be derived individually or in combination using any number of techniques. In an embodiment, the original graph is dense when compared to the trimmed (pruned) spatial graph described herein. To generate the spatial graph, the graph layout generator receives context 1404, vehicle state 1406, graph model 1408, and model parameters 1410 as input. In an embodiment, the graph layout generator 1402 also receives feedback 1418. Based on this input, the graph layout generator 1402 generates a spatial graph (referred to as the original graph) that depicts the underlying model, taking into account the input.

[0136] Context 1404 is applicable to vehicles (e.g., Figure 1 A vehicle (100) is given a set of external conditions, which can be represented by any number of variables. For example, a map provides context such as a representation of the environment, such as the applicable physical area or roads. Positioning provides context such as determining the location within the environment, while perception enables the determination of other data in the environment (including objects in the physical space that the vehicle cannot share, e.g., Figure 10The context of objects 1008a-1008b, etc. In embodiments, the graph layout can be generated statically (e.g., using fixed road geometry and driving statistics), dynamically (e.g., using the current dynamic state of all other road users and dynamic changes in lane boundaries from construction zones), or by any combination thereof. For example, in static graph layout generation, the graph generated by the graph layout generator is separate from the actual road conditions. In dynamic graph layout generation, road conditions determine the graph generation.

[0137] Vehicle state 1406 is also used as input to graph layout generator 1402. Vehicle state 1406 is a set of internal conditions applicable to the vehicle. Vehicle state includes, for example, attitude (i.e., x, y, heading), speed, acceleration, jerk, and angular velocity. Furthermore, vehicle state includes the vehicle's orientation, its relative angle to the road, current acceleration and speed, horizontal jerk, and lateral jerk. In an embodiment, vehicle state 1402 is used to determine various aspects of the spatial structure output by graph layout generator 1402. For example, the vehicle's speed may lead to the removal of physically infeasible edges from the spatial structure, such as high-curvature edges. Typically, a higher-speed vehicle cannot successfully navigate high-curvature edges because the speed will cause the vehicle to physically be unable to maintain a path. The spatial structure generated by graph layout generator 1402 for a higher-speed vehicle will remove high-curvature edges from the graph that cannot be successfully or safely traversed. Conversely, spatial structures for lower-speed vehicles include high-curvature edges, because lower speeds allow the vehicle to navigate tighter curves.

[0138] Graph model 1408 is also provided as input to graph layout generator 1402. The graph model represents multiple data points that represent the edges and vertices of a graph (such as graphs 1000, 1302, and 1312). This data can be obtained using any number of techniques. For example, the graph model is handcrafted. In a handcrafted model, various parameters are adjusted based on an interpretation of how a vehicle operates along a road. The model is then propagated over the space to be covered. In a parametric model, parameters are defined to determine the resulting model. The parameters of the parametric model (parametrically modified model) will take into account the surrounding environment (e.g., Figure 1The environment 190) is mapped to a set of nodes and connected edges. In an embodiment, the graph model 1408 is sample-based, in which a set of vertices and edges that can be used to successfully traverse the space are determined. Additionally, driving data is obtained from human drivers, and this data can be sampled to extract the graph model. In an embodiment, human driver data is randomly sampled to generate the graph model. Other techniques used to determine the graph model 1408 include control techniques for providing uniform coverage, and the use of nonlinear programs created to generate the graph model. In an embodiment, a machine learning-based model is executed to generate the graph model.

[0139] Model parameters are defined for graph models 1410. Model parameters are specific to various types of graph models 1408. For example, model parameters include specific samples (e.g., samples relative to locations on a map or a portion of a graph), sample counts, sample density, minimum or maximum distances between nodes, and minimum or maximum branching factors, etc. Typically, model parameters characterize the spatial aspects of a particular graph model.

[0140] The graph output by the graph layout generator 1402 is input to the edge labeling assignment 1412. For ease of description, the graph output by the graph layout generator 1402 is a spatial structure referred to as the original graph. During edge labeling assignment, the edges of the original graph are labeled. Each of these multiple edges is classified as either useful or useless for labeling the original graph. The usefulness or uselessness of the edges is based at least in part on a distance metric 1416. In an embodiment, the distance metric 1416 is a motion prediction model. In an example, the distance metric 1416 is a probability distribution as described in Covernet: Multimodal behavior prediction using trajectorysets; T. Phan-Minh, EC. Grigore, F. A. Boulton, O. Beijbom, and E. M. Wolff; Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pp. 14074-14083 (2020).

[0141] Typically, distance metric 1416 is a multimodal probabilistic prediction of the future state of a vehicle. Distance metric 1416 calculates a distribution of possible trajectories based on the original graph and driving log 1414 (including the current state of the vehicle as described by vehicle state 1406). Distance metric 1416 measures the proximity of edges to samples of driving data (e.g., driving log 1414). For example, the distance metric takes at least a portion of the driving log trajectory and graph edges as input and outputs a measurement of the proximity of that edge to that portion of the driving log trajectory. In other words, the distance metric calculates the overlap between an edge and a corresponding portion of the driving log trajectory. In an embodiment, the measurement of the proximity of an edge to a corresponding portion of the driving log trajectory is averaged for all corresponding portions of the trajectory in driving log 1414. In an embodiment, the measurement of the proximity of an edge to a portion of the driving log trajectory is the average distance between the edge and the corresponding portion of the driving log trajectory. Driving log 1414 includes real-world driving data (such as past states of all vehicles, pedestrians, cyclists, etc.) and a map. Distance metric 1416 is applied to the driving log 1414 and the original graph output by the graph layout generator 1402 to generate a distribution of possible trajectories with associated probabilities (e.g., likelihoods). Specifically, each edge of the original graph is associated with a probability based on a comparison with the corresponding portion of the driving log 1414.

[0142] For example, using the current state of the vehicle, the original map, and the driving log, a dynamic trajectory set is calculated based on a distance metric. The distance metric generates a dynamic set of possible trajectories based on the current state of the vehicle. Edges from the original map output by the graph layout generator 1402 that largely overlap with corresponding portions of the driving log 1414 result in lower distance metrics because the average distance between the edges and corresponding portions of the driving log is lower due to the higher level of overlap. Therefore, edges with an average distance metric less than a predetermined threshold are marked as useful edges. Edges from the original map output by the graph layout generator 1402 that have an average distance metric greater than a predetermined threshold are marked as useless.

[0143] exist Figure 14 In the example, edges of the original graph output by graph layout generator 1402 are labeled as useful or useless based on a motion prediction model. However, in this embodiment, other factors are analyzed to determine whether edges of the graph output by the graph layout generator are useful. These other factors can be used to override the classification of edges as useful or useless. Other factors can also be analyzed in conjunction with the output of the motion prediction model. For example, an edge is considered useful if it reaches or satisfies a threshold applied to edges in a spatial graph. Edges are also considered useful based on road geometry, including lane centerlines.

[0144] The original graph generated by graph layout generator 1402 and the labels derived by edge label assignment 1412 are provided as input to edge discriminator 1420. At edge discriminator 1420, edges are pruned from the received graph based on the labels. Specifically, if an edge is marked as useless during edge label assignment 1412, that edge is pruned from the graph at edge discriminator 1420. By pruning edges marked as useless, edges that have a relatively high average distance from the edges of the original graph to real-world driving data (e.g., driving log 1414) are ignored. Edges that do not substantially overlap with the sample data are also ignored.

[0145] In one embodiment, the edge discriminator 1420 trims the edges of the original graph based on one or more factors. For example, edges that never match a driving data sample (e.g., driving log 1414) are removed. In another example, the planning module (e.g., Figure 4 The planning module 404 specifies a maximum number of K edges for the simplified graph. In this example, the K connecting edges with the lowest distance metrics are retained in the simplified graph, and the remaining edges are removed. In another example, the K connecting edges that result in the highest overall performance of the graph are retained. Specifically, when multiple edges have high distance metrics, removing one or more edges with high distance metrics may change other factors associated with the remaining edges retained in the final graph. In an embodiment, adaptive edge removal is performed via an iterative algorithm that removes one or more edges and re-evaluates the performance of the edges in the remaining graph. In this embodiment, as edges are adaptively removed from the graph, the simplified graph is selected by choosing the graph with the highest performance.

[0146] The simplified graph is input into evaluation 1422. Pruning the original graph involves strategically removing or reducing edges and nodes from the original spatial graph generated by graph layout generator 1402. Pruning is based on a predetermined graph size, predetermined performance, or any combination thereof. In an embodiment, a machine learning model is used for the pruned graph. Evaluation 1422 determines the actual performance of the simplified graph in light of the performance of the complete original graph generated by graph layout generator 1402. In an embodiment, performance is a measure of accuracy or precision associated with the ability of the pruned graph to replicate human expert trajectories, such as those found in driving log 1414. Thus, in an embodiment, performance is a measure of routes (such as those found in driving log 1414) that can be accurately reproduced using the pruned graph for motion planning. In an embodiment, performance is a measure of the accuracy with which the trajectory generated using the pruned graph replicates the true ground trajectory found in driving log 1414. Graph performance includes measures of the edge types that occur most frequently during motion planning, or of the areas where the simplified graph is used more frequently in motion planning. In an embodiment, the performance of the pruned graph is determined by performing a motion planning module using the simplified graph as input. The strength of this discrimination is adjusted based on driving log 1414 when removing edges from the original graph.

[0147] In feedback loop 1418, graph layout generator 1402 is provided with edge statistics calculated by edge labeling assignment 1412 and graph performance of the pruned graph from evaluator 1422. Edge statistics include (but are not limited to) probabilities associated with edges, such as those determined by distance metric 1416, usefulness based on edge type, or any combination thereof. In the example, graph performance is quantified by data indicating that a certain type of edge or region of the simplified graph is used more frequently in motion planning than other types of edges or regions. This provides positive feedback to graph layout generator 1402 for specific types of edges or other graph data that should be used more frequently in the generation of the original graph compared to other types of features or graph data. Feedback loop 1418 iterates over different original graphs generated by graph layout generator 1402 to improve the graph performance of the resulting simplified graph based on specific metrics used to label the edges of the graph. In the example, preferred edge types are biased as input to graph layout generator 1402 based on statistics and the actual performance of the pruned graph. The edge types of the bias layout generate multiple candidate graph layouts, and the graph layout among the multiple candidate graph layouts is selected as the final original graph to be labeled and pruned as described herein. Given an original graph identical to the original graph output by the graph layout generator 1402, the second feedback loop 1424 adjusts the discrimination of the edge discriminator 1420 based on graph performance. In an embodiment, graph performance is used to iteratively modify the amount or type of edges removed from the original graph at the edge discriminator 1420.

[0148] When compared with the original map generated by the graph layout generator 1402, the final trimmed map achieves large coverage with a smaller or simplified map size. The spatial map is generated based on context and specific graph model parameters and can be adapted or mapped to roads. The spatial structure is input to the planning module and used to achieve a high-quality planning solution. In this way, the motion planning module is no longer trained or otherwise modified against the trimmed map generated by this technique. In one embodiment, a large amount of driving data is used to develop or train the trimmed map, and the trimmed map is trained offline. In other embodiments, the trimmed map is trained or developed in real time.

[0149] Figure 14 The block diagram is not intended to indicate that system 1400 includes Figure 14 All of the components shown. Instead, system 1400 may include fewer or... Figure 14Additional components not shown (e.g., additional graph layout generator inputs, statistics, tags, feedback loops, etc.). System 1400 may include any number of additional components not shown, depending on the specific implementation details. Furthermore, any of the graph layout generator, edge assignment, distance metric, edge discriminator, evaluation, and other described functions may be implemented partially or entirely in hardware and / or a processor. For example, the functions may be implemented using an application-specific integrated circuit, in logic implemented in a processor, in logic implemented in a dedicated graph processing unit, or in any other means.

[0150] Determining the useful edges

[0151] Figure 15 This is a diagram of the spatial structure 1500. In Figure 15 In the example, the spatial structure is generated by graph layout generator 1402 ( Figure 14 The spatial structure 1500 is shown as a roughly rectangular set of nodes 1504 connected by multiple edges 1506, but the structure 1500 is adapted to the curvature of the corresponding road as described above.

[0152] Typically, planning modules (e.g., Figure 4 The planning module 404 takes the spatial structure as input and creates paths covered by a graph. In an embodiment, this technique incorporates expert driving data (e.g., Figure 14 The driving log 1414) and the original diagram (e.g., by Figure 14 The graph output by the graph layout generator 1402 is matched to label edges and derive statistics. The most useful parts of the graph in replicating the behavior of human expert driving data are identified. Furthermore, parts of the graph that do not contribute to performance or cannot replicate the behavior of human expert driving data are also identified. As described above, the graph is pruned based on statistics derived at least in part from a distance metric. In embodiments, the degree of simplification can be controlled by a predetermined graph size or by predetermined performance.

[0153] The vehicle 1502 is shown on the left side of the space structure 1500. Figure 15 In the example, spatial structure 1500 is a graph. This graph includes multiple nodes 1504 and edges 1506. Driving log sample 1508 represents a sample of driving data from a human expert. From driving log 1414 ( Figure 14Driving log sample data is obtained. Driving log sample 1508 includes segment lines 1508A, 1508B, and 1508C. To determine useful edges in the graph, the sequence of human expert driving data represented by segment line 1508 that most closely matches edge 1506 is identified as useful. In the example, edges whose distance to human expert driving data is below a predetermined threshold are marked as useful, and edges whose distance is above the predetermined threshold are marked as useless. As illustrated, the sequence of edges 1510 in the graph most closely matches segment line 1508B. For ease of description, a small portion of the edges is shown as corresponding to segment line 1508B of driving log sample 1508. However, correspondence can occur throughout the entire space, and the illustration of the spatial structure 1500 should not be considered restrictive.

[0154] For each driving log sample, starting from the root node of the spatial structure and the corresponding position of the sample, the sample is compared with the edges of the original spatial structure. At the root node, unreachable edges are ignored. The cumulative cost of replicating the sample is calculated for each path by performing a search along the reachable edges from the root node, where the search is guided by the sample points along the driving log sample. In this way, the cost of each reachable edge marked as useful along possible paths is calculated. The path with the lowest cost can be used to guide the selection of the optimal edge sequence considering a specific driving sample. This process is repeated for multiple driving log samples. When calculating the optimal edge sequence considering the driving log samples, edges considered unreachable are removed. This technique does not select individual edges for inclusion in the pruned graph based on distance from the driving log sample. Instead, this technique selects edge sequences considering real-world driving samples, where each edge sequence is compared with other sequences and irrelevant sequences are removed.

[0155] In this embodiment, a distance metric enables the determination of the proximity of an edge sequence to a real-world trajectory. Edges are labeled based on their ability to be included in an edge sequence that most closely generates or replicates the current sample. In this embodiment, multiple factors are used to determine the best, optimized edge sequence corresponding to each driving log sample. These factors include, but are not limited to, for example, a count of the number of times an edge is used across all samples when it is used in the edge sequence to replicate human driving behavior, or a count of how far the edge is from the sample. Additionally, for samples that are not well covered as other samples, edges that are similar to the poorly covered driving log samples are given higher weights. This occurs, for example, in cases of extreme routes or other special, uncommon maneuvers. In this embodiment, rare edges are injected into the trimmed graph before identifying paths from a first point to a second point on the trimmed graph. The injection of rare edges provides a representation of extreme routes or other special, uncommon maneuvers in the trimmed graph.

[0156] This technique enables higher quality coverage and human-like behavior. The resulting pruned graph is a high-coverage sparse spatial structure from which the planning module can output higher-quality paths with human-level performance. Driving logs are used to adjust the pruned graph for optimizations to frequently occurring scenarios. Furthermore, this technique enables coverage of rare edge cases such as smooth maneuvering and collision avoidance. Using a large amount of driving data for sampling is also sufficient to cover rare edge cases including complex movements. Therefore, large driving datasets yield high-quality solutions. Planning time is also improved by reducing the space of possible options and focusing on alternatives that produce better planning performance. This technique directly optimizes both quality and performance. In an embodiment, this technique is executed offline on a server using hundreds or thousands of human-hours of driving data. The simplified graph is accessed by the planning module for real-time use.

[0157] Figure 15 The spatial structure is not intended to indicate that spatial structure 1500 will include Figure 15 All components shown. Instead, space structure 1500 may include fewer or... Figure 15 Additional components not shown (e.g., additional samples, edges, nodes, etc.). Spatial structure 1500 may include any number of additional components not shown, depending on the specific implementation details.

[0158] Driving data-driven spatial planning

[0159] Figure 16 This is a flowchart of the spatial planning process 1600, which implements driving data-guided processing. At block 1602, at least one processor is used to generate the spatial structure (e.g., Figure 13 The spatial structure includes edges (e.g., Figure 1302). Figure 13 The edge (1306) connects multiple nodes. At least some nodes and edges represent the path of the navigation vehicle from the first point to the second point.

[0160] At box 1604, at least one processor is used to mark the edges of the spatial structure (e.g., Figure 14 Edge labeling assignment 1412). In an embodiment, labeling includes assigning useful or useless labels to each edge based on a distance metric and a predetermined threshold. In an embodiment, corresponding statistics for the respective edges are derived, at least in part, based on a distance metric. In the example, edge statistics include, but are not limited to, those derived from a distance metric (e.g., ...). Figure 14 The distance metric (1416) determines the probability associated with an edge, the usefulness based on the edge type, or any combination thereof.

[0161] At box 1606, the spatial structure is pruned by removing one or more edges from the spatial structure based on the edge labels or corresponding statistics (e.g., Figure 14 The edge discriminator 1420 simplifies the graph based on a predetermined graph size, predetermined performance, or any combination thereof. In an embodiment, the predetermined graph size is measured by the number of edges K in the graph, the graph's storage requirements, or any combination thereof. In an embodiment, the performance of the pruned graph is a measure of accuracy or precision associated with the graph's ability to replicate human expert trajectories, such as those found in driving logs.

[0162] At box 1608, a path from the first point to the second point on the trimmed map is identified. In this embodiment, the path represents the lowest-cost path based on data sampled from human expert driving log data. At box 1610, the vehicle is navigated according to the path from the first point to the second point on the trimmed map.

[0163] Figure 16 The flowchart is not intended to indicate that the boxes in example process 1600 will be executed in any order or that all boxes will be included in every case. Furthermore, depending on the specific implementation details, any number of additional boxes (not shown) may be included within example process 1600.

[0164] Figure 17 This is a process flowchart (1700) that provides feedback during driving data-driven spatial planning. At box 1702, the driving log and pruned map (e.g., ...) are evaluated. Figure 14 Evaluation 1422). In an embodiment, the trimmed graph is trimmed based on markers and statistics prior to evaluation (e.g., Figure 14 Edge discriminator 1420).

[0165] At box 1704, the edges removed from the trimmed graph are iteratively adjusted based on the corresponding performance. In the embodiment, the driving log (e.g., Figure 14 The driving log 1414) is reproduced on the trimmed map, and based on the performance of the trimmed map (e.g., Figure 14 The feedback loop 1424 is used to adjust the degree to which one or more edges are removed from the trimmed graph. Typically, reproducing a driving log path using a trimmed graph involves using a planning module to recreate the driving log path as it appears in the driving log. Additionally, the degree to which one or more edges are removed from the graph is adjusted based on a measurement of the trimmed graph's coverage of the driving log path; this is based on modifying the edges removed from the trimmed graph according to its performance.

[0166] At box 1706, the trimmed graph with the highest performance is selected for path planning. In this embodiment, the edges that result in the highest overall performance of the trimmed graph are retained. Adaptive edge removal is performed via an iterative algorithm that removes one or more edges and re-evaluates the performance of edges in the remaining graph. In this embodiment, as edges are adaptively removed from the graph, a simplified graph is selected by choosing the graph with the highest performance. Therefore, the trimmed graph is iteratively trimmed based on statistical data. In this embodiment, the degree of simplification is based on a predetermined graph size, predetermined performance, or any combination thereof.

[0167] In this embodiment, based on the statistics and actual performance of the current pruned graph, the preferred edge type is biased as input to the graph layout generator (e.g., Figure 14 Feedback loop 1418). For example, a graph layout generator (e.g., Figure 14 The graph layout generator 1402 is provided with edge markers and statistics (e.g., Figure 14 The edge label assignment 1412) and the performance of the trimmed graph (e.g., Figure 14 The evaluation (1422) is used as input to generate the original graph.

[0168] Figure 17 The processing flowchart is not intended to indicate that the boxes in example processing 1700 will be executed in any order or that all boxes will be included in every case. Furthermore, depending on the specific implementation details, any number of additional boxes (not shown) may be included within example processing 1700.

[0169] In the preceding description, embodiments of the invention have been described with reference to numerous specific details, which may vary from implementation to implementation. Therefore, the specification and drawings should be considered illustrative rather than restrictive. The sole and exclusive indication of the scope of the invention, and what the applicant expects to be the scope of the invention, is the literal and equivalent scope of the claims published from this application in the specific form of the claims, including any subsequent amendments. Any definitions of terms expressly set forth herein for inclusion in such claims should be taken as meaning as such terms are used in the claims. Furthermore, when the term “comprising” is used in the preceding specification or appended claims, what follows that phrase may be an additional step or entity, or a sub-step / sub-entity of a previously stated step or entity.

Claims

1. A method for a vehicle, comprising: obtaining, using at least one processor, a spatial structure comprising a plurality of nodes connected by edges, wherein at least a portion of the nodes and edges represent paths for navigating the vehicle from a first point to a second point; labeling, using the at least one processor, edges of the spatial structure as useful based at least in part on a measure of proximity of copying of the edges from driving data samples of a driving log, wherein an edge is useful when a number of times the edge is found in the driving data samples satisfies a threshold, and an edge sequence is useful when a distance between the edge sequence and the driving data samples satisfies a predetermined threshold; pruning, using the at least one processor, the spatial structure by removing at least one edge from the spatial structure according to respective labels of the edges, wherein a degree of the removing is based on a predetermined graph size, a predetermined performance, or any combination thereof, to obtain a pruned graph; adjusting the degree of removing the at least one edge from the spatial structure according to a measure of coverage of paths in the driving log by paths recreated on the pruned graph; and navigating, using the at least one processor, the vehicle according to a path from the first point to the second point representing useful edge sequences on the pruned graph.

2. The method of claim 1, wherein, generating the spatial structure comprises: biasing edge types of the spatial structure according to statistical data and actual performance of the pruned graph, wherein biasing edge types of a graph layout generates a plurality of candidate graph layouts, and selecting a graph layout of the plurality of candidate graph layouts as the spatial structure.

3. The method of claim 1, comprising: labeling, using the at least one processor, edges of the spatial structure as useless based at least in part on the measure of proximity of copying of the edges from driving data samples of the driving log; and pruning, using the at least one processor, the spatial structure by removing edges of the spatial structure labeled as useless to obtain the pruned graph. The pruning comprises removing edges from the spatial structure according to one or more factors.

4. The method of claim 1, wherein, The pruning comprises removing one or more edges from the spatial structure according to edge statistics.

5. The method of claim 1, wherein, The edge represents a series of adjacent locations forming a line without width between the first point and the second point.

6. The method of claim 1, wherein, The edge represents an area of locations forming a bar between the first point and the second point.

7. The method of claim 1, wherein, 8. The method of claim 1, comprising injecting rare edges into the pruned graph before identifying paths from the first point to the second point on the pruned graph.

9. A vehicle, comprising: at least one computer-readable medium storing computer-executable instructions; at least one processor configured to execute the computer-executable instructions, the execution performing the method of claim 1.

10. A non-transitory computer-readable storage medium comprising at least one program for execution by at least one processor of a first apparatus, the at least one program comprising instructions which, when executed by the at least one processor, cause the first apparatus to perform the method of claim 1. ​ 11. A method for a vehicle, comprising: evaluating, using at least one processor, a pruned graph comprising a plurality of nodes connected by edges, wherein edges of the pruned graph are labeled as useful based on distance and driving data samples from a driving log, and a degree to which at least one edge is removed from a spatial structure for generating the pruned graph is based on a measure of coverage of paths in the driving log by paths recreated on the pruned graph; iteratively adjusting, using the at least one processor, edges removed from the pruned graph based on respective performances of the pruned graph; selecting, using the at least one processor, a remaining pruned graph having a highest performance, wherein respective performances of the pruned graph are evaluated at the iterative adjustment; and navigating, using the at least one processor, the vehicle according to a path from a first point to a second point on the selected remaining pruned graph.

12. The method of claim 11, comprising: generating a spatial structure comprising a plurality of nodes connected by edges, wherein a preferred edge type is biased according to statistical data and actual performance of the pruned graph; labeling edges as useful or not useful based on distance and a driving log; pruning the spatial structure by removing edges labeled as not useful to obtain a final pruned graph; and navigating the vehicle according to a path from the first point to the second point on the final pruned graph.

13. The method of claim 11, wherein, labeling edges of a spatial structure as useful comprises calculating respective average distances of edges of the spatial structure and respective portions of trajectories in the driving log, wherein edges having respective average distances satisfying a first predetermined threshold are labeled as useful.

14. The method of claim 11, wherein, iteratively adjusting edges removed from the pruned graph based on respective performances of the pruned graph comprises: removing one or more edges from the pruned graph; and re-evaluating respective performances of edges in the remaining graph.

15. The method of claim 11, comprising removing edges from the pruned graph to meet a predetermined graph size, wherein, the predetermined graph size is measured by a number of edges in a graph, a storage requirement of a graph, or any combination thereof.

16. The method of claim 11, wherein, the path from the first point to the second point on the pruned graph is a lowest cost path, wherein a cost of a path is a value representing resources spent when navigating the vehicle on a respective path.

17. The method of claim 11, wherein, the performance of the pruned graph is an accuracy of edges in replicating driving log trajectories.

18. A vehicle, comprising: at least one computer-readable medium storing computer-executable instructions; at least one processor configured to execute the computer-executable instructions, the execution performing the method of claim 11.

19. A non-transitory computer-readable storage medium comprising at least one program for execution by at least one processor of a first apparatus, the at least one program comprising instructions, which, when executed by the at least one processor, cause the first apparatus to perform the method of claim 11.

20. A computer program product comprising a program for causing a computer to perform the method of any one of claims 1-8 and 11-17.

Images