Learning in Lane-Level Route Planners

Abstract

Lane level route planning includes obtaining lane level information of a road, the road including a first lane and a second lane, the lane level information including first lane information associated with the first lane and second lane information associated with the second lane, converting the lane level information into probabilities for a state transition function, receiving a destination, and obtaining a policy as a solution of a model using the state transition function.

Description

[Technical field]

[0001] The present disclosure relates generally to route planning, and more particularly, to lane-level route planning for autonomous vehicles. [Background technology]

[0002] Route planning applications and systems (collectively, route planners) can be in-vehicle systems or handheld devices. A user (e.g., a driver) provides a destination, and the routing application calculates a route from the user's current location to the destination.

[0003] A route may be overlaid on a map and may include (e.g., show) roads and turns that a user can follow to reach a destination. That is, a route may include turn-by-turn directions to a destination. Typically, a route includes roads and turns, and it is up to the user (e.g., the driver) to decide which lanes to travel on a particular road and when to switch lanes, such as at an exit or intersection.

[0004] A conventional route planner may warn or notify a user of an upcoming turn (e.g., an exit) within a predetermined distance (e.g., 2 miles) and it is up to the user to decide how to maneuver out of their current lane to make the turn. Such route planners do not know what lane the host vehicle is in and simply notify the user within a predetermined distance, which may force the user to make an unsafe maneuver out of their current lane to get to the location of the turn.

[0005] Such route planning applications may plan deterministically at the road level and do not consider lane level information / planning. Route planning may be modeled as a classical planning problem that seeks to minimize route time.

[0006] For at least these reasons, road-level, rather than lane-level, route planning may not be appropriate (e.g., sufficient) for autonomous driving because the planning map in such route planning applications may be at a level of abstraction beyond that required for autonomous driving, which may have no, or at best minimal, driver involvement. Summary of the Invention

[0007] A first aspect is a method for lane level route planning, the method including obtaining lane level information for a road, the road including a first lane and a second lane, the lane level information including first lane information associated with the first lane and second lane information associated with the second lane, converting the lane level information into probabilities for a state transition function, receiving a destination, and obtaining a policy as a solution of a model using the state transition function.

[0008] A second aspect is an apparatus for lane level route planning, including a processor configured to obtain lane level information for a road, the road including a first lane and a second lane, the lane level information including first lane information associated with the first lane and second lane information associated with the second lane, and transform the lane level information into a state transition function used to obtain a route to a destination.

[0009] A third aspect is a non-transitory computer-readable storage medium including executable instructions that, when executed by a processor, facilitate performance of operations including: receiving lane-level information from a plurality of vehicles traveling on a road, deriving a navigation map from the lane-level information, the navigation map including a state transition graph and a reward function, the lane-level information including at least two of environmental information, vehicle information, or human information, and transmitting the navigation map to the vehicles, where the vehicles use the navigation map to derive a lane-level route.

[0010] These and other aspects, features, elements, implementations and embodiment variations of the methods, apparatus, procedures and algorithms disclosed herein are described in further detail below.

[0011] Various aspects of the methods and apparatus disclosed herein will become more apparent with reference to the following description and examples provided in the drawings, in which like reference numerals refer to like elements. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an example of a vehicle in which aspects, features, and elements disclosed herein may be implemented.

[0013] [Diagram 2] FIG. 1 is an example diagram of a portion of a vehicle traffic and communication system in which aspects, features, and elements disclosed herein may be implemented.

[0014] [Diagram 3] FIG. 1 is a flow chart diagram illustrating an example of a technique for route planning according to an embodiment of the present disclosure.

[0015] [Figure 4] FIG. 13 illustrates an example of a map according to an implementation of the present disclosure.

[0016] [Diagram 5] FIG. 1 illustrates an example of a preference ranking graph of objectives in accordance with implementations of the present disclosure.

[0017] [Figure 6] FIG. 1 illustrates an overview of learning and using a navigation map according to an implementation of the present disclosure.

[0018] [Figure 7] FIG. 2 is a flow chart diagram illustrating an example of a technique for learning a navigation map for route planning according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] As mentioned above, route planners typically provide routes without lane information. Such routes may be planned based on a connected graph of Global Positioning System (GPS) coordinates, which is too abstract to be used for autonomous driving. Autonomous driving requires lane-specific information to make lane changes and decisions, and to plan for contingencies in case an obstacle occurs that makes it impossible to follow the route for some reason. In traditional route planner maps, topology, connectivity, and lane information are typically not available. As used herein, autonomous driving includes performing vehicle driving control to automatically execute a host vehicle, as compared to manual control by a human driver or teleoperator, unless the context indicates otherwise.

[0020] Moreover, such route planners may typically only solve for one objective, i.e., time, performing classical route planning by minimizing the time it takes to reach a destination. Some route planners may allow the user to specify road type preferences (e.g., highways instead of local roads, free roads instead of toll roads). However, autonomous driving requires route planning to be performed using different (e.g., more) criteria, constraints, and / or objectives.

[0021] To illustrate using a simple example, in a traditional route planner, if a driver misses an expected turn and exits, the route planner typically performs a "re-routing" operation that takes at least several seconds to complete to identify an alternative route. In the case of autonomous driving, assuming an autonomous vehicle (AV) misses an exit due to congestion, it is unacceptable for the AV to stop in place until the routing application identifies (i.e., re-routes) an alternative route and traverse the contingency route while performing the re-routing operation.

[0022] Rather, a route planner according to the present disclosure can pre-plan for all contingencies. The contingencies can be incorporated into policies as described below. In one example, the contingencies can be planned taking into account previous probabilities. The probabilities can be based on historical data of lane segment traffic. The probabilities can include probabilities of successful lane changes, probabilities of failure to change lane based on traffic, traffic history probabilities, or other probabilities. The lane level route planner can also plan routes based on capability levels for autonomous driving on different roads or road segments. Capabilities are further described below. The route planner can also determine (e.g., plan) transitions of control between autonomous driving, a driver, and / or a teleoperator. The lane level route planner uses a navigation map that incorporates probabilities.

[0023] To reiterate, a route planner according to the present disclosure determines a lane-level route to a destination, including locations along the route where the AV will be controlled to make lane changes. That is, lane changes and possibly obstacles along the route can be planned for by the route planner. Additionally, the route planner can have contingency plans for all possible routes to a goal (e.g., destination), assuming successes and failures, and not just obtain a deterministic plan that minimizes cost (e.g., time to destination).

[0024] Real-world deployment of autonomous vehicles may require balancing multiple objectives simultaneously. Some objectives may include time to destination, desired user comfort (e.g., bumpy roads vs. smooth roads), desired user road speed, road navigation capabilities (e.g., whether the vehicle has the ability to traverse roads or lanes), transition points between geofenced areas, and / or other objectives. In the case of electric or hybrid electric vehicles, additional objectives may relate to battery efficiency and / or stopping points for recharging. Lane-level route planning according to the present disclosure can perform such multi-objective planning.

[0025] In aspects of the present disclosure, a navigation map that includes lane-level information may be learned, and then a lane-level route may be generated using the navigation map.

[0026] Conventional autonomous vehicle route planners may not learn about multiple state features or objectives. In contrast, in lane-level route planning according to implementations of the present disclosure, the navigation map may incorporate at least some environmental information, vehicle information, human information, lesser, more, or other information, or combinations thereof, as further described herein, about multiple lane segments of a road lane.

[0027] The risk-aware multi-objective lane-level route planning described herein requires customized learning tuned to the probabilistic and specific distinct objective reward values ​​of each lane as considered in the multi-objective space of autonomous connected vehicles, as described herein.

[0028] Multi-objective learning may be advantageous over single-objective learning, at least because more concerns can be taken into account by the lane-level route planner, making autonomous driving more adaptive than a single-objective planner. When multiple objectives are possible, the lane-level route planner can learn (or be taught by the user) about good (e.g., desired, comfortable) routes, traffic patterns, user preferences, capabilities, etc. For example, the user can directly encode preferences with precisely corresponding objectives.

[0029] The details of lane-level route planning are described herein first with reference to the environment in which it may be implemented.

[0030] FIG. 1 illustrates an example of a vehicle in which aspects, features, and elements disclosed herein may be implemented. In the illustrated embodiment, the vehicle 100 includes various vehicle systems. The vehicle systems include a chassis 110, a powertrain 120, a controller 130, and wheels 140. Additional or different combinations of vehicle systems may be used. For simplicity, the vehicle 100 is shown to include four wheels 140, but any other propulsion device, such as one or more propellers or treads, may be used. In FIG. 1, lines interconnecting elements, such as the powertrain 120, the controller 130, and the wheels 140, indicate that information, such as data or control signals, forces, such as power or torque, or both information and power, may be transferred between each element. For example, the controller 130 may receive power from the powertrain 120 and communicate with the powertrain 120, the wheels 140, or both, to control the vehicle 100, which may include accelerating, decelerating, steering, or otherwise controlling the vehicle 100.

[0031] 1 includes a power source 121, a transmission 122, a steering device 123, and an actuator 124. Any other element or combination of elements of the powertrain may be included, such as a suspension, a drive shaft, an axle, or an exhaust system. Although shown separately, wheels 140 may be included in the powertrain 120.

[0032] The power source 121 includes an engine, a battery, or a combination thereof. The power source 121 may be any device or combination of devices that operates to provide energy, such as electrical energy, thermal energy, or kinetic energy. In one example, the power source 121 includes an engine, such as an internal combustion engine, an electric motor, or a combination of an internal combustion engine and an electric motor, that operates to provide kinetic energy as motive power to one or more of the wheels 140. Alternatively or additionally, the power source 121 may include a potential energy device, such as one or more dry batteries, such as nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), solar cells, fuel cells, or any other device capable of providing energy.

[0033] The transmission 122 receives energy, such as kinetic energy, from the power source 121 and sends the energy to the wheels 140 to provide motive power. The transmission 122 may be controlled by the controller 130, the actuator 124, or both. The steering device 123 may be controlled by the controller 130, the actuator 124, or both and control the wheels 140 to steer the vehicle. The actuator 124 may receive signals from the controller 130 and may operate or control the power source 121, the transmission 122, the steering device 123, or any combination thereof to operate the vehicle 100.

[0034] In the illustrated embodiment, the controller 130 includes a positioning device 131, an electronic communication device 132, a processor 133, a memory 134, a user interface 135, a sensor 136, and an electronic communication interface 137. A smaller number of these elements may be present as part of the controller 130. Although shown as a single device, any one or more elements of the controller 130 may be incorporated into any number of separate physical devices. For example, the user interface 135 and the processor 133 may be incorporated into a first physical device and the memory 134 may be incorporated into a second physical device. Although not shown in FIG. 1, the controller 130 may include a power source 1210, such as a battery. Although shown as separate elements, the positioning device 131, the electronic communication device 132, the processor 133, the memory 134, the user interface 135, the sensor 136, the electronic communication interface 137, or any combination thereof, may be incorporated into one or more electronic devices, circuits, or chips.

[0035] The processor 133 may include any device or combination of devices now existing or later developed capable of manipulating or processing signals or other information, including optical processors, quantum processors, molecular processors, or combinations thereof. For example, the processor 133 may include one or more special purpose processors, one or more digital signal processors, one or more microprocessors, one or more controllers, one or more microcontrollers, one or more integrated circuits, one or more application specific integrated circuits, one or more field programmable gate arrays, one or more programmable logic arrays, one or more programmable logic controllers, one or more state machines, or any combination thereof. The processor 133 may be operatively coupled to one or more of the positioning device 131, the memory 134, the electronic communication interface 137, the electronic communication device 132, the user interface 135, the sensor 1360, and the powertrain 136. For example, the processor may be operatively coupled to the memory 134 via a communication bus 138.

[0036] Memory 134 includes any tangible, non-transitory computer usable or computer readable medium capable of, for example, holding, storing, transmitting or carrying machine-readable instructions or any information associated therewith for use by or in connection with processor 133. Memory 134 may be, for example, one or more solid-state drives, one or more memory cards, one or more removable media, one or more read-only memories, one or more random access memories, one or more disks, including hard disks, floppy disks, optical disks, magnetic or optical cards, or any type of non-transitory medium suitable for storing electronic information, or any combination thereof. For example, memory may be one or more read-only memories (ROMs), one or more random access memories (RAMs), one or more registers, low power DDR (LPDDR) memories, one or more cache memories, one or more solid-state memory devices, one or more magnetic media, one or more optical media, one or more magneto-optical media, or any combination thereof.

[0037] The communications interface 137 may be a wireless antenna as shown, a wired communications port, an optical communications port, or any other wired or wireless device capable of interfacing with a wired or wireless electronic communications medium 150. Although Figure 1 illustrates communications interface 137 communicating over a single communications link, the communications interface may be configured to communicate over multiple communications links. Although Figure 1 illustrates a single communications interface 137, the vehicle may include any number of communications interfaces.

[0038] The communication device 132 is configured to transmit or receive signals over a wired or wireless electronic communication medium 150, such as via a communication interface 137. Although not explicitly shown in FIG. 1, the communication device 132 may be configured to transmit, receive, or both over any wired or wireless communication medium, such as radio frequency (RF), ultraviolet (UV), visible light, optical fiber, wired line, or a combination thereof. Although FIG. 1 shows a single communication device 132 and a single communication interface 137, any number of communication devices and any number of communication interfaces may be used. In some embodiments, the communication device 132 includes a Dedicated Short Range Communications (DSRC) device, an On-Board Unit (OBU), or a combination thereof.

[0039] Positioning device 131 may determine geographic information such as the longitude, latitude, altitude, heading, or speed of vehicle 100. In one example, the GPS includes a Global Positioning System (GPS) device such as a Wide Area Augmentation System (WAAS)-enabled National Marine Electronics Association (NMEA) device, a radio triangulation device, or a combination thereof. Positioning device 131 may be used to obtain information representing, for example, the current heading of vehicle 100, the current location of vehicle 100 in two or three dimensions, the current angular heading of vehicle 100, or a combination thereof.

[0040] The user interface 135 includes any device capable of interfacing with a person, such as a virtual or physical keypad, touch pad, display, touch display, heads-up display, virtual display, augmented reality display, tactile display, feature tracking device such as an eye-tracking device, speaker, microphone, video camera, sensor, printer, or any combination thereof. The user interface 135 may be operatively coupled to the processor 133 as shown, or to any other element of the controller 130. Although shown as a single device, the user interface 135 may include one or more physical devices. For example, the user interface 135 may include both an audio interface for voice communication with the person, and a touch display for visual and touch-based communication with the person. The user interface 135 may include multiple displays, such as multiple physically separate devices, multiple defined portions of a single physical device, or a combination thereof.

[0041] The sensors 136 are operable to provide information that can be used to control the vehicle. The sensors 136 may be an array of sensors. The sensors 136 may provide information about the current operating characteristics of the vehicle 100, including vehicle operation information. The sensors 136 may include, for example, speed sensors, acceleration sensors, steering angle sensors, traction-related sensors, brake-related sensors, steering wheel position sensors, eye tracking sensors, seating position sensors, or any sensor or combination of sensors operable to report information about some aspect of the current dynamic situation of the vehicle 100.

[0042] Sensors 136 may include one or more sensors operable to obtain information about the physical environment surrounding vehicle 100, such as operating environment information. For example, one or more sensors may detect road geometry, such as lanes, and obstacles, such as fixed obstacles, vehicles, and pedestrians. Sensors 136 may be or include one or more video cameras, laser sensing systems, infrared sensing systems, acoustic sensing systems, or any other suitable type of on-board environment sensing device or combination of devices, now known or later developed. In some embodiments, sensors 136 and positioning device 131 are combined.

[0043] Although not separately shown, the vehicle 100 may include a trajectory controller. For example, the controller 130 may include a trajectory controller. The trajectory controller may be operable to obtain information describing the current state of the vehicle 100 and a planned route for the vehicle 100, and to determine and optimize a trajectory for the vehicle 100 based on this information. In some embodiments, the trajectory controller may output signals operable to control the vehicle 100 such that the vehicle 100 follows a trajectory determined by the trajectory controller. For example, the output of the trajectory controller may be an optimized trajectory that may be provided to the powertrain 120, the wheels 140, or both. In some embodiments, the optimized trajectory may be a control input, such as a set of steering angles, each steering angle corresponding to a time or position. In some embodiments, the optimized trajectory may be one or more paths, lines, curves, or combinations thereof.

[0044] One or more of the wheels 140 may be steering wheels that are pivoted to a steering angle under the control of the steering device 123, propulsion wheels that are given a torque to propel the vehicle 100 under the control of the transmission 122, or steering and propulsion wheels that may both steer and propel the vehicle 100.

[0045] Although not shown in FIG. 1, the vehicle may include additional devices or elements not shown in FIG. 1, such as an enclosure, a Bluetooth® module, a frequency modulation (FM) radio device, a near field communication (NFC) module, a liquid crystal display (LCD) display device, an organic light emitting diode (OLED) display device, a speaker, or any combination thereof.

[0046] Vehicle 100 may be an autonomous vehicle that is autonomously controlled without direct human intervention to navigate a portion of a transportation network. Although not separately shown in FIG. 1 , the autonomous vehicle may include an autonomous vehicle controller that provides routing, navigation, and control of the autonomous vehicle. The autonomous vehicle controller may be integrated with other devices of the vehicle. For example, controller 130 may include an autonomous vehicle controller.

[0047] If present, the autonomous vehicle controller may control or operate the vehicle 100 to move through a portion of the vehicle traffic network according to the current vehicle operating parameters. The autonomous vehicle controller may control or operate the vehicle 100 to perform a defined operation or maneuver, such as parking the vehicle. The autonomous vehicle controller may generate a travel route from a starting point, such as the current location of the vehicle 100, to a destination based on the vehicle information, the environmental information, the vehicle traffic network information representing the vehicle traffic network, or a combination thereof, and may control or operate the vehicle 100 to move through the vehicle traffic network according to the route. For example, the autonomous vehicle controller may output the travel route to a trajectory controller to operate the vehicle 100 to move from the starting point to the destination using the generated route.

[0048] Figure 2 is an example diagram of a portion of a vehicle traffic and communication system in which aspects, features and elements disclosed herein may be implemented. The vehicle traffic and communication system 200 may include one or more vehicles 210 / 211, such as the vehicle 100 shown in Figure 1, that may travel through one or more portions of a vehicle traffic network 220 and communicate through one or more electronic communication networks 230. Although not explicitly shown in Figure 2, the vehicles may travel through off-road areas.

[0049] Electronic communications network 230 may be a multiple access system providing communications, such as, for example, voice, data, video, messaging, or combinations thereof, between vehicles 210 / 211 and one or more communications devices 240. For example, vehicles 210 / 211 may receive information, such as information describing vehicular traffic network 220, from communications device 240 via network 230.

[0050] In some embodiments, vehicles 210 / 211 may communicate via a wired communication link (not shown), a wireless communication link 231 / 232 / 237, or any combination of any number of wired or wireless communication links. As shown, vehicles 211 / 211 communicate via a land-based wireless communication link 231, via a non-land-based wireless communication link 232, or via a combination thereof. Land-based wireless communication link 231 may include an Ethernet link, a serial link, a Bluetooth link, an infrared (IR) link, an ultraviolet (UV) link, or any link capable of providing electronic communications.

[0051] Vehicles 210 / 211 may communicate with other vehicles 210 / 211. For example, a host or target vehicle 210 may receive one or more autonomous vehicle-to-vehicle messages, such as a basic safety message (BSM), from a remote or target vehicle (RV) 211 via a direct communication link 237 or via a network 230. The remote vehicle 211 may broadcast a message to the host vehicle within a defined broadcast range, such as 300 meters. In some embodiments, the host vehicle 210 may receive the message via a third party, such as a signal repeater (not shown) or another remote vehicle (not shown). Vehicles 210 / 211 may transmit one or more autonomous vehicle-to-vehicle messages periodically based on a defined interval, such as 100 milliseconds.

[0052] The automatic vehicle-to-vehicle messages may include vehicle identification information, geospatial state information such as longitude, latitude or altitude information, geospatial position accuracy information, motion state information such as vehicle acceleration information, yaw rate information, speed information, vehicle heading information, braking system state information, throttle information, steering angle information or vehicle routing information, or vehicle operating state information such as vehicle size information, headlight state information, turn signal information, wiper state information, transmission information or any other information or combination of information related to the transmitting vehicle state. For example, the transmission state information may indicate whether the transmitting vehicle's transmission is in neutral, parked, forward or reverse.

[0053] The vehicle 210 may communicate with the communication network 230 via an access point 233. The access point 233, which may include a computing device, is configured to communicate with the vehicle 210, the communication network 230, one or more communication devices 240, or a combination thereof, via wireless or wired communication links 231 / 234. For example, the access point 233 may be a base station, a base transceiver station (BTS), a Node-B, an enhanced Node-B (eNode-B), a Home Node-B (HNode-B), a wireless router, a wired router, a hub, a relay, a switch, or any similar wired or wireless device. Although shown here as a single device, the access point may include any number of interconnection elements.

[0054] Vehicle 210 may communicate with communication network 230 via satellite 235 or other non-terrestrial communication devices. Satellite 235, which may include a computing device, is configured to communicate with vehicle 210, communication network 230, one or more communication devices 240, or combinations thereof, via one or more communication links 232 / 236. Though shown here as a single unit, the satellite may include any number of interconnecting elements.

[0055] Electronic communications network 230 is any type of network configured to provide voice, data, or any other type of electronic communications. For example, electronic communications network 230 may include a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a mobile or cellular telephone network, the Internet, or any other electronic communications system. Electronic communications network 230 uses communications protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Internet Protocol (IP), Real-time Transport Protocol (RTP), Hypertext Transport Protocol (HTTP), or combinations thereof. Although shown here as a single device, electronic communications network may include any number of interconnected elements.

[0056] The vehicle 210 may identify a portion or state of the vehicular traffic network 220. For example, the vehicle includes at least one on-board sensor 209, such as the sensor 136 shown in FIG. 1, which may be or include a speed sensor, a wheel speed sensor, a camera, a gyroscope, an optical sensor, a laser sensor, a radar sensor, an acoustic sensor, or any other sensor or device capable of determining or identifying a portion or state of the vehicular traffic network 220, or a combination thereof.

[0057] Vehicle 210 may navigate one or more portions of vehicular traffic network 220 using information communicated over network 230, such as information representative of traffic network 220, one or more onboard sensors 209, or a combination thereof.

[0058] For simplicity, one vehicular traffic network 220, one electronic communications network 230, and one communications device 240 are shown in Figure 2, although any number of networks or communications devices may be used. Vehicular traffic and communications system 200 may include devices, equipment, or elements not shown in Figure 2. Although vehicle 210 is shown as a single unit, the vehicle may include any number of interconnected elements.

[0059] Although vehicle 210 is shown communicating with communication device 240 via network 230, vehicle 210 may communicate with communication device 240 via any number of direct or indirect communication links. For example, vehicle 210 may communicate with communication device 240 via a direct communication link, such as a Bluetooth® communication link.

[0060] 3 is a flow chart diagram illustrating an example of a technique for route planning according to an embodiment of the present disclosure. The technique 300 may be implemented partially or fully in a host vehicle, which may be the vehicle 100 shown in FIG. 1, one of the vehicles 210 / 211 shown in FIG. 2, a semi-autonomous vehicle, any other vehicle that may include driver assistance features, or a vehicle that is manually controlled by a driver or the like. The technique 300 may be implemented as instructions (e.g., operations) stored in a memory, such as the memory 134 of FIG. 1. The instructions may be executed by a processor, such as the processor 133 of FIG. 1. The technique 300 may be implemented using dedicated hardware or firmware. Multiple processors, memories, or both may be used.

[0061] The technique 300 may be implemented partially or completely by a lane level route planner. The lane level planner may be implemented using a processor including a CPU (Central Processing Unit), a memory, and an input / output device as described with reference to FIG. 1 and the like. A computer program that causes the processor to function as the lane level route planner may be installed and executed on the processor. This allows the processor to function as the lane level route planner. Note that, although an example in which the lane level route planner is implemented by software will be described here, it goes without saying that the lane level route planner may also be implemented by dedicated hardware prepared to execute each information processing described here.

[0062] In step 302, the technology 300 receives a destination. In one example, a user (e.g., a driver) can provide the destination via a user interface of a routing application. In one example, a user can provide the destination to a routing application that may be running on the user's portable device. In another example, a vehicle is running a service (e.g., a robotaxi service) and the destination can be provided to the technology 300 via a dispatch process, whereby the vehicle travels to the destination, which may be a customer or baggage pick-up location or a customer or baggage drop-off location. Other ways of providing the destination to the technology 300 are also possible. The destination may be an address, a landmark, a venue, or other destination type. The technology 300 can convert the received destination into a set of GPS coordinates.

[0063] In step 304, the technique 300 obtains a lane level route to the destination using a map (i.e., a navigation map or a lane level navigation map). The lane level route includes a transition (i.e., an action for the transition) from a first segment of a first lane of a road to a second segment of a second lane of the road. In one example, the destination may be transmitted (wirelessly) and provided to a server (e.g., a cloud-based server), such as the communication device 240 of FIG. 2, which calculates the lane level route. An example of a map and lane level route is described with respect to FIG. 4.

[0064] 4 is a diagram illustrating an example of a map according to an implementation of the present disclosure. Map 400 shows that a destination is on road segment 401. Map 400 includes road 402 that includes three lanes, lanes 404A-404C. The lanes in map 400 are divided into lane segments. The lane segments are indicated by numbered segments in map 400. For example, lane 404A includes lane segments 406A-406C. Vehicle 403 is currently on lane segment 406A and is heading toward the destination of lane segment 401. Lane segment 406A may be an intermediate lane segment between a starting point (not shown) of vehicle 403 and the destination.

[0065] In one example, all road segments may have the same length (e.g., 100 meters, 200 meters, or other segment length). In one example, the length of a lane segment may depend at least in part on the speed along the road of the lane segment. For example, at high speeds, the lane segment may have a longer length, such as 500 meters. In one example, the length of some lane segments may be adjusted. That is, for example, when generating a first lane level route, the first lane level route may be generated by a portion of the lane segments having a first length, and when generating a second lane level route, the second lane level route may be generated by a portion of the lane segments having a second length. The length of the lane segments may be changed based on the speed on the road, the time of day (e.g., rush hour vs. weekend), socio-geographic region (e.g., school zone, residential area), weather (e.g., sunny vs. snow), road type (e.g., highway vs. city road), more, less, other criteria, or combinations thereof. For example, as visually illustrated, lane segment 406E is shorter than lane segment 406C, but lane segment 406D is longer than lane segment 406C.

[0066] In map 400, lane segments may have identification numbers (IDs). For example, lane segments 406A-406E have IDs 1, 2, 3, 7, and 13, respectively. At least some of the lane segments may be associated with metadata. In one example, the metadata may include one or more classifications (e.g., designations). For example, map 400 may indicate (using different shading) that lane segments with IDs 1, 2, and 3 are high-speed segments, lane segments with IDs 4, 5, 6, 7, and 11 are standard-speed (e.g., speed-limited) segments, lane segments with IDs 8, 9, and 10 are comfort (e.g., low-speed) segments, lane segments with IDs 12, 13, and 15 are city lane segments, and lane segments with IDs 14, 16, and 17 are manual-driving lane segments. These classifications are not necessarily mutually exclusive. For example, an urban lane segment may be a manually driven lane segment.

[0067] A comfort lane segment may be a lane segment along which traffic may travel at a speed within -10 miles per hour (or other threshold speed) of the speed limit for the roadway of the lane segment. A manually driven lane segment is a lane segment that cannot be traversed autonomously or cannot be traversed autonomously with sufficient confidence. Such lane segments must be traversed under manual driving control, such as by a driver or teleoperator. As such, the lane-level route planner may include an action to notify the user (e.g., the driver) a certain time prior to the lane segment that the user should take over manual driving control of the vehicle.

[0068] In one example, obtaining a lane level route may include obtaining a policy for traversing lane segments from an origin location to a destination. The policy may include lane transitions. More precisely, as described below, the policy provides actions given that the vehicle is in a current lane segment. In one example, the policy may be a deterministic policy. In another example, the policy may be based on a probabilistic model.

[0069] In route planning, minimizing the time to a destination is the typical objective of a route planner, but the lane-level route planner according to the present disclosure can obtain a lane-level route by optimizing for several objectives, as further described below.

[0070] Lane-level routes may be obtained using one type of Markov decision process (MDP) model. Lane-level route planning may be modeled as a topological partially observable Markov decision process (TPOMDP) ​​or its fully observable subclass, the topological Markov decision process (TMDP). TPOMDP and TMDP allow for the incorporation of additional objective measures, such as maximizing safety, smoothness, and / or other people's preferences, into the objectives of a typical partially observable Markov decision process model (POMDP) ​​or Markov decision process (MDP), such as minimizing travel time or travel distance. Solving a TPOMDP model may be similar to solving a TMDP model, except that TPOMDP is based on belief states (i.e., probabilities), which represent the probability of each state, followed by the observation probabilities corresponding to the generation of observations for each state. Other model types are possible. For example, the model can be a scalarized multiobjective Markov decision process (MOMDP), a scalarized partially observable multiobjective Markov decision process (MOPOMDP), a constrained Markov decision process (CMDP), or a constrained partially observable Markov decision process (CPOMDP).

[0071] To illustrate, without loss of generality, a user may indicate a preference for slow lanes, so that the route planner can incorporate a "comfort" objective into the calculation of how to reach the destination. In another example, a user may additionally or alternatively indicate a preference for lanes that minimize energy consumption. For example, if traffic in some lane segments tends to be slow and require more energy consumption, the route planner may prioritize other road segments. As another example, smooth roads are prioritized by the route planner over roads with many uphill and downhill slopes, since such roads tend to require more energy consumption. As another example, a road with many charging stations may be prioritized over another road with fewer charging stations.

[0072] Without loss of generality, the model (eg, TMDP or TPOMDP) ​​can be described in stochastic shortest path (SSP) structural terms (eg, costs instead of rewards, and goals as target states that self-loop upon arrival).

[0073] A connected graph (V, E, R) is a set of tuples 〈S, A, T, C, E, δ, s0, s g 〉. Each vertex v∈V may be the GPS coordinate of a point where a routing decision may be made. Thus, a vertex may be the GPS coordinate of an intersection, a lane change location (e.g., from lane segment ID9 to lane segment ID11), a merge point, etc. To illustrate, in map 400, a vertex v may be the coordinate of a particular point of a lane segment. For example, the particular point may be the midpoint of a lane segment. Each edge e∈E of the connectivity graph may describe a directed graph of vertices of length or travel time R(e). For example, in the connectivity graph below map 400, the vertex corresponding to lane segment ID9 is connected to lane segments ID10 and 11.

[0074] Next, we outline the formal model TMDP. ​​As mentioned before, this model is formally expressed as a tuple 〈S, A, T, C, E, δ, s0, s g〉. S may be a finite set of states or vertices v∈V. A is the action space at each vertex for selecting a subsequent lane segment (in other words, a subsequent vertex). To illustrate without loss of generality, the action space A may include the actions "move left", "move right", "move straight", or "move diagonally". However, more, less, other actions, or combinations thereof are possible. For example, the action space A may include actions to inform the user to take over manual control, to switch to manual control, to inform the user that control is switching to automatic control, to switch to automatic control, etc.

[0075] s0 is the initial state, which may be the current position of the vehicle when the route planner calculates the lane-level route. g is the goal state, which can be the destination. g When the goal state is reached, no matter what action is taken, the goal state is maintained, and the goal state s g All actions performed with have a cost of 0.

[0076] T (i.e., T: S×A×S→[0,1]) may be a state transition function that describes the probability of a subsequent state s′∈S occurring after performing an action a∈A in state s∈S. Thus, T is a state transition function that describes the probabilistic success / failure of each action. The state transition function T(s,a,s′)=Pr(s′|s,a) is the probability s′ of a subsequent (lane segment or vertex) given that action a was performed in state s. In one example, the state transition function may include respective probabilities of at least some (e.g., each) of the adjacent vertices (e.g., lane segments) of a vertex (e.g., lane segment). To illustrate, for a lane segment ID1 (i.e., lane segment 406A), the state transition function T may include respective transition probabilities for some of the transitions from the lane segment ID1 to the lane segment ID4, the transitions from the lane segment ID1 to the lane segment ID2, and / or the transitions from the lane segment ID1 to the lane segment ID5. In some examples, the state transition function may include a transition probability to a more distant neighbor (eg, from lane segment ID1 to lane segment ID3).

[0077] As such, using the map to obtain a lane-level route to the destination may include obtaining a policy that provides an action to control the AV, when the AV is on a first lane segment of the map, to move into a second lane segment adjacent to the first lane segment.

[0078] The probabilities may be derived based on criteria including speed limits, lane lengths of edge e, AV specifications, time and location, traffic congestion information, more criteria, fewer criteria, other criteria, or combinations thereof. In one example, these probabilities (and / or the criteria used to derive the transition probabilities) may be learned, as described further below.

[0079]

number

number

[0080] The objectives may be arranged in a preference ranking that follows a directed acyclic graph (DAG) E. An example of a preference ranking graph is illustrated in Figure 5. Each objective may have a non-negative slack δ : e → R+, which describes how much a user is willing to "spend" on the value of one objective to improve the value of another objective. Thus, one or more objectives are related by a preference ranking that includes a slack variable.

[0081] A solution to the model may be a policy π:S→A. Under policy π, an action a (i.e., π(s)) is selected for state s. That is, policy π may indicate that action π(s)∈A should be executed at state s. Policy π is a method to move from state s to a destination (i.e., goal state s) according to policy π. g ) the expected cumulative cost to reach π A value function V that can represent (s) π : S → C. That is, the value function is calculated for each intermediate state s from the starting state to the goal state. i The expected cost (i.e., value) of

[0082] Optimal policy π * minimizes the expected cumulative cost. Formally, for the initial state s0, the expectation can be given by equation (1).

number

[0083] In formula (1), S t and A t Let π denote the state and action random variables at time step t, respectively. In general, to solve a model, an appropriate policy must exist. An appropriate policy π has the following properties: (1) there exists a policy that reaches the goal with probability 1, and (2) every state that does not reach the goal with probability 1 incurs infinite cost.

[0084] In step 306 of FIG. 3, the technique 300 controls the AV to travel the lane-level route. Returning to FIG. 4, the arrows in FIG. 4 indicate actions under the determined policy π. Arrow 408A indicates the action "if the state is equal to the lane segment of ID1, perform the right action, so that the vehicle ends up in a state equal to the lane segment of ID4." In this case, "perform the action" means that "the vehicle is controlled" by changing the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, etc.

[0085] The determined policy π incorporates (e.g., includes) a contingency route. Thus, the resulting lane-level route includes the contingency route. To illustrate, if the vehicle is in lane segment ID9, the policy indicates that the vehicle is controlled to move to lane segment ID11. However, if the vehicle is unable to take lane segment ID11 and enters lane segment ID10, the policy indicates "perform left action" to move the vehicle to lane segment ID6 and then to lane segment ID7. Thus, controlling the AV to traverse the lane-level route may include controlling the AV according to the contingency route in response to failing to transition from the first segment to the second segment.

[0086] FIG. 5 illustrates an example preference ranking graph 500 of objectives according to an implementation of the present disclosure. As described above, the objectives of the model may be topologically ordered. The topologically ordered constraints may satisfy the slack of the first belief or all beliefs against the preceding objectives. A lane-level route planner can reason about the objectives to compute an ordered policy in the topologically ordered constraint graph. Although three different preference rankings are described, the present disclosure is not limited thereto and other configurations (i.e., preference rankings) are possible.

[0087] The objectives shown in example 500 relate to speed optimization (i.e., bubbles labeled H, such as objectives 512, 538, and 556), comfort optimization (i.e., bubbles labeled C, such as objectives 514, 534, and 554), city driving optimization (i.e., bubbles labeled U, such as objectives 516, 532, and 552), and manual driving optimization (i.e., bubbles labeled M, such as objectives 518, 536, and 558). The illustrated objectives are examples, and other objectives are possible. Furthermore, the illustrated preference ranking graph is a non-limiting example, and other preference ranking graphs including the same or different objectives and topologies are possible. Each of these objectives is associated with a respective semantic label that may be meaningful to a human. As described further herein, a user may use the semantic labels to indicate route preferences for a lane-level route planner.

[0088] Example 510 is an example of a chained preference ranking graph. When using the preference ranking graph of example 510 (i.e., when the objectives are linked in a chain), first the high speed objective is maximized (i.e., objective 512), then the comfort objective (i.e., objective 514) is maximized if it is within the high speed objective's slack 513 (e.g., a slack of 3), then the city objective (i.e., objective 516) is maximized if it is within the comfort objective's slack 515 (e.g., a slack of 1), which has already been constrained to ensure that it is within the first objective's slack 513, and finally the manual driving objective (i.e., objective 518) is maximized if it is within the already constrained city objective's slack 517 (e.g., a slack of 7).

[0089] Example 510 may be interpreted (e.g., understood) as a lane-level route planner preferring the highway (i.e., objective 512), but allowing 3 minutes of extra time (i.e., slack 513) in order for the route planner to select another alternative route that is more comfortable (i.e., objective 514), which also maximizes the comfort objective. So, a user may provide input (e.g., issue a command) that essentially indicates, "Give me the fastest road possible, but allow me a route that is 3 minutes longer than the fastest in order to drive in the more comfortable lane," and similarly for the other objectives in example 510. It should be noted that the unit of measure of the slack variable is the same as the unit of measure of the objective it constrains.

[0090] As described further below, the user can specify desired preferences using verbal commands that the route planner can convert into a preference ranking graph via a graphical user interface (e.g., a drag-and-drop interface, a list, etc.) or via other manners for the user to provide the preferences.

[0091] Example 530 is an example of a sector-shaped preference ranking graph. When the preference ranking graph of example 530 is used (i.e., the objectives have such a sector-like relationship), optimal policies are calculated simultaneously for the city objective (i.e., objective 532), the comfort objective (i.e., objective 534), and the manual driving objective (i.e., objective 536). Then, a final policy is determined that maximizes the high speed objective (i.e., objective 538) subject to the slacks 533, 535, and 537 (i.e., slack values ​​5, 9, and 2, respectively) of the three initial objectives. This is equivalent to taking the intersection of the policy sets of the first three objectives and searching that space for the final objective.

[0092] Example 550 is an example of a directed graph that mixes examples 510 and 530.

[0093] For each objective (e.g., a bubble in example 500), the lane-level route planner obtains a set of routes that satisfy the objective. For example, for objective 552, the lane-level route planner identifies an optimal route (e.g., a minimal route to reach the destination with as many city lanes as possible). A slack of one minute (i.e., slack 553) is allowed to reach the goal. Thus, the route planner can be thought of as providing an envelope of possibilities around reaching the goal. That is, the route planner can identify all routes that are within one minute of the optimal time, and whether they are on city roads or not. Thus, an initial set of routes that satisfy objective 552 within slack 553 is returned.

[0094] A second set of routes meets objective 554 (i.e., stay in lanes marked as comfortable) and is within 6 minutes (i.e., slack 555) of the most comfortable route. Similarly, a third set of routes is obtained that meets the manual driving objective within 9 minutes (i.e., slack 557), and so on. The full set of retrieved routes is passed to objective 556, and routes retrieved from this sink objective must be in all of the sets of retrieved routes. In some situations, a route that meets all constraints and slack may not be possible.

[0095] Note that the incoming arrows to the bubbles in example 500 can be thought of as constraining that objective to be within a given set of routes (or sets, depending on the configuration of the preference ranking graph). Thus, in the absence of a parent arrow (i.e., no incoming arrows), any route that satisfies the objective within any slack (if any) may be selected by the route planner. On the other hand, the selected route(s) must be within the routes provided by the parent objective. Thus, downstream objectives can be thought of as pruning the set(s) of provided (e.g., incoming) routes.

[0096] In one example, instead of a preference ranking graph, a scalarization function can be used to map (e.g., combine, etc.) all rewards (e.g., costs) to a single value. The scalarization function can be used to convert the model / problem into a shortest path optimization problem (SSP) MDP or POMDP. Using the scalarization function, a single value indicating the long-term utility of the next immediate action may be obtained, which combines the expected costs to obtain the single value. Formally, the scalarization function f:C→R is expressed as f(V(s))=V in terms of the single value. f (s), and in terms of cost function f(C(s))=C f (s). In one example, the scalarization function may be a weighted sum of the objectives or a non-linear function of the objectives. Different weights may be used depending on the desired optimization. In one example, lane-level route planning may be modeled as a classical planning (CP) problem by eliminating probabilities.

[0097] Regardless of the model used, lane level route planning may be hierarchical. As can be appreciated, the greater the number of possible lanes, if any, the longer the policy may take to compute. To limit the number of lane segments that the lane level route planner infers, lane level route planning may be performed in a hierarchical manner. For example, for an origin in Miami, Florida, and a destination in Fairbanks, Alaska, the lane level route planner may group US roads into clusters, plan lane level routes between the clusters, and then perform route level planning within each cluster of the route.

[0098] In one example, as discussed above, lane-level route planning may be modeled as a topological partially observed MDP (TPOMDP) ​​and allow for beliefs about, for example, (1) traffic levels in each state, (2) the capabilities of at least one of the drivers and / or vehicles, and / or (3) various objective costs.

[0099] More formally, TPOMDP can be written as a tuple 〈S, A, Ω, T, O, R, E, δ〉.

[0100] S is a finite set of states. A is a finite set of actions. Ω is a finite set of observations. T is the state transition function, where T(s,a,s')=Pr(s'|s,a) is the probability of subsequent state s' given that action a was executed in state s. O is the observation function, where O(a,s',ω)=Pr(w|a,s') is the observation probability ω given that action a was executed and the result was subsequent state s'.

[0101] R is a vector of reward functions, which may be equivalent to the cost vector C discussed above. As discussed above, E is a set of edges for k rewards (e.g., costs), forming a directed acyclic graph with one leaf (i.e., sink) reward vertex, which is reward vertex k, without loss of generality. δ is a function that maps edges e=〈i,j〉∈E to nonnegative slack constraints δ(e)≧0 or equivalently δ(i,j)≧0.

[0102] TPOMDP is a set of world beliefs b∈B⊆Δ |S| where Δ |S| is a probability distribution over state S, and as a standard |S|-simplex. Belief b is a probability distribution over states. Beliefs exist at every time step and for every forward, backward, following, leading, etc. lane segment. For belief b, after taking action a and making observation ω, the next belief b for state s' is ba ω is

number

[0103] As mentioned above, TMDP is a subclass of TPOMDP with Ω=S and O(a,s,s')=1, and an attainable belief b∈B is b(s)=1 for all s∈S.

[0104] The objective of the infinite horizon TPOMDP is to maximize the discounted reward expected from an initial belief b0 with a discount factor γ∈[0,1). For a policy π, the expected discounted reward is

number

[0105]

number

number

[0106] In equation (3), R(b,a)=Σ s b(s)R(s,a) and b bπ(b)ω ' follows the belief update in equation (2).

[0107] As described herein, the lane level route planner derives the policies using a navigation map. In one example, the navigation map may be learned. As used herein, learned includes that the navigation map may be evolved. The navigation map may include pre-populated lane information, and as lane level information is obtained from one or more vehicles, the pre-populated lane information may be evolved to reflect (e.g., incorporate) the obtained lane level information. The navigation map may not initially include lane level information, and the lane level information may be built (e.g., populated) based on lane level information received from one or more vehicles.

[0108] The lane level information may be received continuously and the navigation map may be periodically updated using newly received lane level information. The navigation map may be a vehicle specific navigation map (e.g., deployed in a particular vehicle), and the navigation map may be updated based on lane level information from the vehicle itself.

[0109] Additionally or alternatively, the navigation map may be learned based on lane-level information received (such as at a server) from many vehicles. The lane-level information may be with the server, which may be a cloud-based server, so that multiple vehicles may then receive accurate initial estimates for time (before actually driving on the road), preferences for popular routes (in new areas), and capability information for where the fleet of vehicles (of vehicles sharing the same code / hardware) should be planning to travel along the route.

[0110] Lane level information may be obtained from many vehicles. Vehicles owned by a particular customer may generate lane level information. A deployed fleet of vehicles (such as those providing taxi services, delivery services, and other services) may generate lane level information. Lane level information from a vehicle may be generated using data received from sensors on the vehicle. In one example, raw sensor data may be sent to a server, and the server may generate lane level information using the raw sensor data.

[0111] In this manner, the navigation map may include learned historical driving patterns and characteristics for different lanes and different lane segments of different roads. The historical driving patterns may be either for a particular vehicle, a particular driver of a particular vehicle, or the aggregated learned historical driving patterns of multiple vehicles.

[0112] The driving patterns of one or more drivers may be learned and used for lane-level route planning. The navigation map may include information about road conditions. Thus, the learned navigation map may leverage / incorporate the driver's driving behavior history and / or characteristics of lanes (more specifically, lane segments) to enable lane-level route planning.

[0113] Lane-level information from multiple connected vehicles (e.g., connected to a cloud-based server) may be combined into a shared navigation map. Each connected vehicle may send lane-level information to a server, such as one or more communication devices 240, which can aggregate all of the obtained lane-level information to obtain a navigation map. Multi-vehicle multi-objective learning (i.e., lane-level information from multiple connected vehicles) can significantly speed up the learning of a single vehicle, allowing the vehicle to learn route values ​​(e.g., lane-level information) before driving on the road itself.

[0114] Over time, the server may receive information regarding many lane segments that many drivers have traveled in the area, as well as roads that a particular vehicle has traveled on. The lane level information may also include an identification of the vehicle platform (e.g., Nissan Leaf, Nissan Sentra, Nissan Rogue, etc.). In some examples, the lane level information may also include information (e.g., preferences, etc.) of the particular driver for whom the lane level information was generated. Using the received lane level information trace, the server can separate the information into those that are dependent on the vehicle (i.e., vehicle specifications), driving style, road (e.g., including lane and lane segment conditions), and / or other criteria to create unique navigation maps, each specific to a particular vehicle, a particular vehicle platform, a particular driver, a particular driving style, etc.

[0115] As described above, regarding the lane segments of the lanes of the road, and through learning, at least some environmental information, vehicle information, human information, lesser, more, other information, or combinations thereof may be learned. The learned information may be converted into probabilities. The probabilities may be incorporated into or used by state transition functions, as described herein. The probabilities may also be used in setting objectives regarding the route (e.g., policy) to be calculated.

[0116] The environmental information may include one or more of traffic patterns, pedestrian patterns, lane segment impassability information by vehicles (e.g., other AVs), weather information, load information, more, less, other environmental information, or a combination thereof. The vehicle information may include one or more of average speed information per lane segment, energy use (e.g., battery use for electric vehicles, gasoline and battery use for hybrid vehicles, or gasoline use for internal combustion engine vehicles), more, less, other vehicle information, or a combination thereof.

[0117] The human information may include one or more of a preferred route, a preferred lane, a request for control transfer (such as from automated to manual or from manual to automated), a capability model update with driver override, more, less, other human information, or a combination thereof. The request for control transfer may include the lane segment for which control transfer is requested. The request for control transfer may include the planned subsequent lane segment and which subsequent lane segment the human driver has taken.

[0118] As the vehicle is traversing a route (whether based on the current state of a navigation map or not setting the route via a lane level planner), the vehicle's sensors can be used to gather information that is converted into lane level information, i.e. the vehicle can be thought of as being in shadow mode and monitoring the environment via its sensors.

[0119] Data from the sensors may be used to identify, for example, without limitation, whether a road (and thus lanes and lane segments) is urban or freeway, whether a lane segment is difficult to navigate, whether traffic is congested, the level of density, etc. Difficulty may be identified based on the amount of driver override, i.e., the number of times an automated driving decision is overridden or modified by a human, or the number of times a human instructs the automated control to control the vehicle in a different manner than suggested by the automated control.

[0120] The load information may indicate the level of stress that a user (e.g., a vehicle occupant) may be under given a selected lane change decision. To illustrate, a particular route may require the vehicle to move three lane segments to the left in a relatively short time or distance in a congested or fast traffic area. Such a situation may be stressful for the user. Such load information may be used to select a lane segment transition point much earlier than the congested area, or to select a route that avoids the stressful situation altogether.

[0121] Using lane level information, counts may be performed such that the counts can be converted to probabilities. For example, without loss of generality, a count of traffic density may be converted to a probability of moving from one lane segment to another, such as the time it takes to travel one lane segment of a road to move to the next, a count of the historical number of times traffic was light and heavy on a lane segment may be used to determine a probability of traffic density, a count of the number of times speeds on a lane segment may be used to classify the lane segment as being high speed, a count of the number of times speeds on a lane segment were within -10 miles of the speed limit may be used to determine the probability of the lane segment being a comfort lane segment, and so on.

[0122] The navigation map may be constructed from probabilities, i.e., probabilities may be incorporated into the navigation map. The navigation map is a transition graph as described with respect to FIG. 4. In deriving the policy, the next action (e.g., move straight to the next lane segment, move to the next lane segment to the left, etc.) may be determined based on lane level information (e.g., probabilities). To illustrate without loss of generality, the next action may be determined probabilistically (or deterministically) based on the probabilities of adjacent lane segments described herein, including, for example, the probability that the adjacent lane segment is congested based on whether the current lane segment is congested.

[0123] At least a portion of the lane-level information of the navigation map may be used to set lane-level route planner objectives. For purposes of illustration, and not limitation, as discussed above, the environment may include difficult passages, and if the objective (e.g., set by the user) is to maximize automated driving, the route planner will avoid lanes (or lane segments) that are difficult to pass. That is, the route planner policies are calculated to avoid lane segments that are difficult to pass.

[0124] Consider objectives regarding time to destination (which may include or incorporate speed limits, road length, stop signals, traffic density, whether the lane segment is a freeway, etc.), preferences (which may include at least some user preferences such as comfort, city roads as opposed to highways, energy consumption, etc.), and capabilities (which may include whether the lane segment can be traversed automatically or manually).

[0125] For illustration, but not by way of limitation, whenever a vehicle travels a route, the time to automatically traverse a particular lane segment may be recorded with or without a full stop, such that a route may be planned using at least some of the time, priority, and capacity objectives. Similarly, the time to manually traverse a particular lane segment (i.e., when a human is driving) may also be recorded. Thus, four average durations may be recorded. The average time may be used to define the TMDP state transition for the self-loop (stop). The average time may be used to define the reward / cost of traversal time, for example, based on the state of automated driving (e.g., control) versus manual driving.

[0126] Additionally, whenever a human drives a lane segment, subsequent lane segments may be recorded. The subsequent lane segments may be used to define rewards / costs for preferences in the range [0, 1], which may be the proportion of time the human drives a road or lane segment.

[0127] Additionally, whenever a human overrides the automatic control or the vehicle under automatic control gets stuck (i.e., does not progress beyond a first threshold, such as 1 minute or other time threshold), a -1 may be recorded as the first value, and otherwise (e.g., when the automatic control is not overridden and the vehicle does not get stuck), a 0 may be recorded as the second value. The average of the recorded first and second values ​​may be used as a measure of competence. The inverse may be used to learn the competence of a human driver. For example, when under manual control, the vehicle does not move beyond a second time threshold (e.g., 1 minute or other time threshold), a -1 may be recorded as the third value. For example, when the driver does not follow a prohibited next action identified by the lane-level route planner, a -1 may be recorded as the fourth value. For example, when the driver seamlessly proceeds into an adjacent lane segment, a 0 may be recorded as the fifth value. The average of the third, fourth, and fifth values ​​may be recorded as a measure of the human's competence to perform a next action when the vehicle is in the current lane segment.

[0128] As previously mentioned, semantic labels may be used to define the objective of the route calculated by the lane-level route planner. Examples of semantic labels include those described with respect to FIG. 5, such as “comfort” (and / or related semantic concepts), “urban” (and / or related semantic concepts such as “highway”), “highway” (and / or related semantic concepts such as “manual driving” and / or related semantic concepts such as “automated driving”). Fewer, more, other semantic labels, or combinations thereof are possible. To illustrate, if a user indicates a preference for comfortable roads, the lane-level route planner calculates the policy taking into account lane segments labeled as “comfort.”

[0129] Note that the mapping from purposes to semantic labels does not have to be a one-to-one mapping, so a semantic label may combine several purposes, or one purpose may be used to define one or more semantic labels.

[0130] In one aspect of the disclosed implementation, multi-objective deep reinforcement learning (MODRL) may be used. In reinforcement learning or deep reinforcement learning, a deep neural network (DNN) may be used to learn the policy. Alternatively or additionally, a DNN may be used to learn the value function. Therefore, reinforcement learning techniques may be used instead of planning techniques. In reinforcement learning, the objective may be learned via the DNN from feedback. The feedback may include one or more of the time to traverse a lane segment, human overrides to capabilities, or other feedback. In reinforcement learning, instead of learning probabilities, probabilities and costs are combined into an expected value for reaching the goal (i.e., destination).

[0131] In one aspect, TMDP may derive policies using a navigation map, as described herein. In another aspect, TPOMDP may derive policies using TPOMDP, which may enable beliefs about, for example, but not limited to, the level of traffic on a road, lane, or lane segment; the amount of capacity of a new road, lane, or lane segment; or other lane-level information as described herein.

[0132] FIG. 6 shows an overview 600 of learning and using a navigation map according to an implementation of the present disclosure. The lane level route planner may include a learning module that obtains lane level information. The lane level route planner may run in a vehicle, and the lane level information may be incorporated into a navigation map of the vehicle. The lane level information may be sent to a server, and the server may receive the lane level information from multiple vehicles. The server may incorporate the received lane level information into the navigation map. The navigation map may then obtain a policy (e.g., a route) using the lane level route planner.

[0133] Overview 600 shows three different routes 602, 604, and 606 taken (by the same or multiple vehicles, not shown) from an origin 608 to a destination 610. During travel, lane level information is collected for lane segments, such as lane segment 612. History 614 shows learned lane transitions, and the shading of the lane segments in history 614 indicates learned characteristics of the lane segments. History 614 includes other lane segment information learned as described herein.

[0134] The history of at least some (e.g., all) of the lane segments along at least some (e.g., all) of the route may be combined (e.g., counted) into a navigation map, as described herein. As described herein, multiple intents and other semantic labels may be recorded for each lane segment history, as indicated by box 618.

[0135] Based on the learning, a state transition graph structure (i.e., state transition function T described above) may be generated, as illustrated by transition graph 620. The state transition graph may be used by an SSP MDP or POMDP model, as described herein. A lane-level multipurpose lane-level route planner navigation map is then output (e.g., generated). The lane-level information of the navigation map includes averages of vehicle occupancy of lane segments. For example, the lane-level information may include averages of one or more of speeds, stops, perceived traffic density, battery usage, lane change success rate, etc.

[0136] Based on the learning, a cost (reward) function to be used in the SSP MDP or POMDP model can also be generated. The average of different objectives can be recorded as rewards by observing time, control transfer points, battery usage, speed, etc. as described above. From the reward function, it can be known whether slack is possible between the objectives. For example, if the expected value of the route is 10 minutes longer than the optimal, then a slack of at least 10 is possible. Such reward functions and slack can be used to set a preference ranking, as described with respect to FIG. 5 and shown in preference ranking 622.

[0137] 7 is a flow chart diagram illustrating an example of a technique for learning a navigation map for route planning according to an embodiment of the present disclosure. The navigation map is used by a lane-level route planner. The technique 300 may be implemented partially or fully in an apparatus that may be a host vehicle, such as the vehicle 100 shown in FIG. 1, one of the vehicles 210 / 211 shown in FIG. 2, a semi-autonomous vehicle, another vehicle that may include driver assistance features, or a vehicle that is manually controlled by a driver, etc. The apparatus may be a server, which may be the communication device 240 of FIG. 2.

[0138] Technique 700 may be implemented as instructions (e.g., instructions) stored in a memory. The memory may be memory 134 of FIG. 1. The memory may be memory of a server. The instructions may be executed by a processor, such as processor 133 of FIG. 1. Technique 700 may be implemented using dedicated hardware or firmware. Multiple processors, memories, or both may be used.

[0139] In step 702, the technique 700 obtains lane level information for a road. The lane level information includes information for different segments of different lanes of a road. For example, a road may include a first lane and a second lane. The lane level information includes first lane information associated with the first lane and second lane information associated with the second lane.

[0140] The lane-level information may be obtained at a host vehicle, and the lane-level information may be used to evolve a navigation map of the host vehicle (e.g., used in, used by, available to the host vehicle). The lane-level information may be obtained (e.g., received) at a server that obtains lane-level information from multiple vehicles. The server may send the evolved navigation map to the vehicles for use by their respective lane-level route planners. The server may use the evolved navigation map to calculate routes (e.g., policies).

[0141] As discussed above, the lane level information may include at least one of environmental information, vehicle information, or human information. The environmental information may include at least one of traffic patterns, pedestrian patterns, or impassability information for the lane segment. The vehicle information may include a user's preferred route and a user's request to transfer control.

[0142] In one example, obtaining lane level information of the road may include recording a first time of automated traversal of the road with stops on the road, recording a second time of automated traversal of the road without stops on the road, recording a third time of manual traversal of the road with stops on the road, and recording a fourth time of manual traversal of the road without stops on the road. In one example, obtaining lane level information of the road may include recording a subsequent lane segment during manual traversal. In one example, obtaining lane level information of the road may include recording a human override of the automated cruise control. Automatic traversal of the road means that a vehicle is automatically controlled to traverse the road. Manual traversal of the road means that a vehicle is manually controlled (e.g., by a human) to traverse the road.

[0143] In step 704, the technique 700 converts the lane level information into a state transition function that can be used to obtain a route to the destination. As described above, the state transition function may be of a TMDP or TPOMDP model. The route to the destination may be, or may be encapsulated in, a policy, as described above. In one example, the policy, i.e., the route, is probabilistic.

[0144] In step 706, the technique 700 receives a destination. For example, a server (or, more specifically, a lane level route planner running on the server) may receive a current location and a destination (e.g., from a user device, a mapping application in the vehicle, etc.) to obtain a route. For example, a lane level route planner running on the device may receive a destination from a user (e.g., a driver or passenger), and the lane level route planner may obtain a route. For example, the lane level route planner may infer a start location for the route based on the current location of the device, etc.

[0145] In step 708, the technique 700 obtains the policy as a solution of the model using the state transition function.

[0146] As used herein, the term "instructions" may include instructions or expressions for performing any method disclosed herein, or any part or parts thereof, and may be implemented in hardware, software, or any combination thereof. For example, instructions may be implemented as information, such as a computer program stored in a memory, that may be executed by a processor to perform any of the methods, algorithms, aspects, or combinations thereof described herein. Instructions or portions thereof may be implemented as a dedicated processor or circuitry, which may include dedicated hardware for performing any of the methods, algorithms, aspects, or combinations thereof described herein. In some implementations, portions of instructions may be distributed across multiple devices or multiple processors on a single device, which may communicate directly or over a network, such as a local area network, a wide area network, the Internet, or combinations thereof.

[0147] As used herein, the terms "example," "embodiment," "implementation," "aspect," "feature," or "element" indicate serving as an example, example, or illustration. Unless expressly stated, any example, embodiment, implementation, aspect, feature, or element is independent of each other example, embodiment, implementation, aspect, feature, or element and may be used in combination with any other example, embodiment, implementation, aspect, feature, or element.

[0148] As used herein, the terms "determining" and "identifying" or any variation thereof includes selecting, ascertaining, calculating, retrieving, receiving, determining, establishing, obtaining, or otherwise identifying or determining in any manner using one or more of the devices shown and described herein.

[0149] As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or" unless otherwise specified or apparent from the context. Furthermore, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless the context clearly indicates a singular reference or unless otherwise specified.

[0150] Furthermore, although for simplicity of description, the drawings and descriptions herein may include a series of steps or stages or sequences, elements of the methods disclosed herein may occur in various orders or simultaneously. Moreover, elements of the methods disclosed herein may occur with other elements not explicitly shown and disclosed herein. Moreover, not all elements of the methods described herein are required to implement a method according to the present disclosure. Although aspects, features, and elements are described herein in certain combinations, each aspect, feature, or element may be used independently or in various combinations with or without other aspects, features, and elements.

[0151] The above aspects, examples and implementations are described to facilitate understanding of the present disclosure and are not intended to be limiting. To the contrary, the present disclosure encompasses various modifications and equivalent arrangements that are within the scope of the appended claims, which should be interpreted broadest possible to encompass all such modifications and equivalent arrangements as are permitted by law.

Claims

1. A method for lane level route planning executed by a processor of a vehicle, comprising: obtaining lane level information for a road indicative of traffic on a lane segment; the road includes a first lane and a second lane; the lane-level information includes first lane information associated with the first lane and second lane information associated with the second lane; The lane-level information includes a lane segment impassability; The impassability is determined based on at least one of a human override or modification of an automated driving decision or a control of the vehicle that differs from a control proposed by an automated control of the vehicle; converting the lane-level information into probabilities for a state transition function, the probability is based on historical data of traffic on the lane segment; and incorporating said probabilities into a navigation map; receiving a destination for routing a host vehicle from an origin location; receiving an objective related to the impassability; and obtaining a policy as a solution of a model using the state transition function based on the navigation map to identify a route from the origin location to the destination and a location along the route at which the host vehicle is controlled to make a lane change, the identified route satisfying the objective related to the impassability; A method comprising:

2. The method of claim 1 , wherein the lane level information is obtained at a host vehicle.

3. The method of claim 1 , wherein the lane level information is obtained from a plurality of vehicles.

4. 1. An apparatus for lane level route planning, comprising: Obtaining lane level information of a road, the road includes a first lane and a second lane; the lane-level information includes first lane information associated with the first lane and second lane information associated with the second lane; Obtaining the lane level information for the road includes: recording a first time of automated travel of said roadway involving a stop on said roadway; recording a second time of automated travel along the roadway without stopping on the roadway; recording a third time of manual traversal of the roadway involving a stop on the roadway; and recording a fourth time of manual travel of said roadway without stopping on said roadway; and converting the lane level information into a state transition function based on a probability calculated using the first time, the second time, the third time, and the fourth time, the state transition function being used to obtain a route from a starting location to a destination; 16. An apparatus comprising: a processor configured to:

5. The apparatus of claim 4 , wherein the lane level information is obtained from a host vehicle.

6. The apparatus of claim 4 , wherein the lane level information is obtained from a plurality of vehicles.

7. The apparatus of claim 4 , wherein the lane level information includes at least two of environmental information, vehicle information, or human information.

8. The apparatus of claim 7 , wherein the environmental information includes at least two of a traffic pattern, a pedestrian pattern, or impassability information for a lane segment.

9. The apparatus of claim 7 , wherein the vehicle information includes a user's preferred route and a control transfer request by the user.

10. Obtaining a route to the destination includes: Obtaining a policy as a solution of a model using said state transition function. The apparatus of claim 4 , comprising:

Images