Explainability and interface design for lane-level routes

Abstract

Providing an explanation in route planning includes determining a route based on at least two objectives received from a user, where a second of the at least two objectives is constrained within a margin value of a first of the at least two objectives, receiving a request from the user for an explanation regarding an action along the route, and providing an explanation to the user, where the explanation describes the extent to which the margin values ​​are violated.

Description

[Technical field]

[0001] The present disclosure relates generally to route planning, and more particularly, to lane-level route planning for autonomous vehicles. [Background technology]

[0002] Route planning applications and systems (collectively, route planners) can be in-vehicle systems or handheld devices. A user (e.g., a driver) provides a destination, and the routing application calculates a route from the user's current location to the destination.

[0003] A route may be overlaid on a map and may include (e.g., show) roads and turns that a user can follow to reach a destination. That is, a route may include turn-by-turn directions to a destination. Typically, a route includes roads and turns, and it is up to the user (e.g., the driver) to decide which lanes to travel on a particular road and when to switch lanes, such as at an exit or intersection.

[0004] A conventional route planner may warn or notify a user of an upcoming turn (e.g., an exit) within a predetermined distance (e.g., 2 miles) and it is up to the user to decide how to maneuver out of their current lane to make the turn. Such route planners do not know what lane the host vehicle is in and simply notify the user within a predetermined distance, which may force the user to make an unsafe maneuver out of their current lane to get to the location of the turn.

[0005] Such route planning applications may plan deterministically at the road level and do not consider lane level information / planning. Route planning may be modeled as a classical planning problem that seeks to minimize route time.

[0006] For at least these reasons, road-level, rather than lane-level, route planning may not be appropriate (e.g., sufficient) for autonomous driving because the planning map in such route planning applications may be at a level of abstraction beyond that required for autonomous driving, which may have no, or at best minimal, driver involvement. Summary of the Invention

[0007] A first aspect is a method for providing explanations in route planning, the method including: determining a route based on at least two objectives received from a user, where a second of the at least two objectives is constrained within a margin value of a first of the at least two objectives; receiving a request from the user for an explanation regarding an action along the route; and providing the explanation to the user, the explanation describing the extent to which the margin value is violated.

[0008] A second aspect is an apparatus for route planning for an AV (Autonomous Vehicle), including a processor configured to receive from a user a first objective for optimizing a route from a start location to a destination, receive from the user a second objective for optimizing the route from the start location to the destination, receive from the user a margin value defining a deviation from the first objective to improve the second objective, determine a route that satisfies the first objective, the second objective, and the margin value, and control the AV to traverse the route.

[0009] A third aspect is a non-transitory computer-readable storage medium including executable instructions that, when executed by a processor, facilitate execution of operations for route planning for an AV, including receiving from a user a first objective for optimizing a route from a starting location to a destination, receiving from the user a second objective for optimizing the route from the starting location to the destination, receiving from the user a margin value defining a deviation from the first objective to improve the second objective, determining a route that satisfies the first objective and the second objective and the margin value, controlling the AV to move from a first lane segment to a second lane segment along the route, receiving from the user a request for an explanation as to why the second lane segment was taken instead of a third lane segment, and providing the explanation, the explanation including respective descriptors of the first objective, the second objective, and the extent to which the margin value is violated.

[0010] These and other aspects, features, elements, implementations and embodiment variations of the methods, apparatus, procedures and algorithms disclosed herein are described in further detail below.

[0011] Various aspects of the methods and apparatus disclosed herein will become more apparent with reference to the following description and examples provided in the drawings, in which like reference numerals refer to like elements. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 illustrates an example of a vehicle in which aspects, features, and elements disclosed herein may be implemented.

[0013] [Diagram 2] FIG. 1 is an example diagram of a portion of a vehicle traffic and communication system in which aspects, features, and elements disclosed herein may be implemented.

[0014] [Diagram 3] FIG. 1 is a flow chart diagram illustrating an example of a technique for route planning according to an embodiment of the present disclosure.

[0015] [Figure 4] FIG. 13 illustrates an example of a map according to an implementation of the present disclosure.

[0016] [Diagram 5] FIG. 1 illustrates an example of a preference ranking graph of objectives in accordance with implementations of the present disclosure.

[0017] [Figure 6] FIG. 1 illustrates an overview of learning and using a navigation map according to an implementation of the present disclosure.

[0018] [Figure 7] FIG. 2 is a flow chart diagram illustrating an example of a technique for learning a navigation map for route planning according to an embodiment of the present disclosure.

[0019] [Figure 8] FIG. 1 is a flowchart illustrating an example of a technique for providing route planning instructions in accordance with an implementation of the present disclosure.

[0020] [Figure 9] FIG. 13 illustrates an example of providing explanations based on multiple objectives according to implementations of the present disclosure.

[0021] [Figure 10] FIG. 13 illustrates an example of a user interface for multi-objective setting of a lane-level route planner in accordance with an implementation of the present disclosure.

[0022] [Figure 11] FIG. 13 illustrates another example of a user interface for multi-objective setting of a lane-level route planner in accordance with an implementation of the present disclosure.

[0023] [Figure 12]FIG. 1 illustrates an example map showing the use of a standard definition (SD) map for route planning in autonomous driving according to an implementation of the present disclosure.

[0024] [Figure 13] FIG. 1 is a diagram of a system for searching for new roads to improve a map in accordance with an implementation of the present disclosure.

[0025] [Figure 14] A figure showing an example of road mapping in an SD map according to an implementation of the present disclosure.

[0026] [Figure 15] FIG. 1 is a flowchart illustrating an example of a technique for route planning for automated driving of an autonomous vehicle in accordance with an implementation of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] As mentioned above, route planners typically provide routes without lane information. Such routes may be planned based on a connected graph of Global Positioning System (GPS) coordinates, which is too abstract to be used for autonomous driving. Autonomous driving requires lane-specific information to make lane changes and decisions, and to plan for contingencies in case an obstacle occurs that makes it impossible to follow the route for some reason. In traditional route planner maps, topology, connectivity, and lane information are typically unavailable. As used herein, autonomous driving includes performing vehicle driving control to automatically execute a host vehicle, as compared to manual control by a human driver or teleoperator, unless the context indicates otherwise.

[0028] Moreover, such route planners typically solve for only one objective: time. That is, they perform classical route planning by minimizing the time it takes to reach a destination. Some route planners may allow users to specify road type preferences (e.g., highways instead of local roads, free roads instead of toll roads). However, autonomous driving requires route planning to be performed using different (e.g., more) criteria, constraints, and / or objectives.

[0029] To illustrate using a simple example, in a traditional route planner, if a driver misses an expected turn and exits, the route planner typically performs a "re-routing" operation that takes at least several seconds to complete to identify an alternative route. In the case of autonomous driving, assuming an autonomous vehicle (AV) misses an exit due to congestion, it is unacceptable for the AV to stop in place until the routing application identifies (i.e., re-routes) an alternative route and traverse the contingency route while performing the re-routing operation.

[0030] Rather, a route planner according to the present disclosure can pre-plan for all contingencies. The contingencies can be incorporated into the policy as described below. In one example, the contingencies can be planned taking into account previous probabilities. The probabilities can be based on historical data of lane segment crossings. The probabilities can include probabilities of successful lane changes, probabilities of failure to change lane based on traffic, traffic history probabilities, or other probabilities. The lane level route planner can also plan routes based on capability levels for autonomous driving on different roads or road segments. Capabilities are further described below. The route planner can also determine (e.g., plan) transitions of control between autonomous driving, a driver, and / or a teleoperator. The lane level route planner uses a navigation map that incorporates probabilities.

[0031] To reiterate, a route planner according to the present disclosure determines a lane-level route to a destination, including locations along the route where the AV will be controlled to make lane changes. That is, lane changes and possibly obstacles along the route can be planned for by the route planner. Additionally, the route planner can have contingency plans for all possible routes to a goal (e.g., destination), assuming successes and failures, and not just obtain a deterministic plan that minimizes cost (e.g., time to destination).

[0032] Real-world deployment of autonomous vehicles may require balancing multiple objectives simultaneously. Some objectives may include time to destination, desired user comfort (e.g., bumpy roads vs. smooth roads), desired user road speed, road navigation capabilities (e.g., whether the vehicle has the ability to traverse roads or lanes), transition points between geofenced areas, and / or other objectives. In the case of electric or hybrid electric vehicles, additional objectives may relate to battery efficiency and / or stopping points for recharging. Lane-level route planning according to the present disclosure can perform such multi-objective planning.

[0033] In aspects of the present disclosure, a navigation map that includes lane-level information may be learned, and then a lane-level route may be generated using the navigation map.

[0034] Conventional autonomous vehicle route planners may not learn about multiple state features or objectives. In contrast, in lane-level route planning according to implementations of the present disclosure, the navigation map may incorporate at least some environmental information, vehicle information, human information, lesser, more, or other information, or combinations thereof, as further described herein, about multiple lane segments of a road lane.

[0035] The risk-aware multi-objective lane-level route planning described herein requires customized learning tuned to the probabilistic and specific distinct objective reward values ​​of each lane as considered in the multi-objective space of autonomous connected vehicles, as described herein.

[0036] Multi-objective learning may be advantageous over single-objective learning, at least because more concerns can be taken into account by the lane-level route planner, making autonomous driving more adaptive than a single-objective planner. When multiple objectives are possible, the lane-level route planner can learn (or be taught by the user) about good (e.g., desired, comfortable) routes, traffic patterns, user preferences, capabilities, etc. For example, the user can directly encode preferences with precisely corresponding objectives.

[0037] Some route planners can only inform the user of general information about the route, such as the number of lanes at the exit, the proximity of the exit, etc. Some other route planners may indicate whether a given route is longer or shorter than the current route by a given number of minutes or some other unit of measurement. However, such route planners typically cannot explain the following: i.e., they do not provide an explanation for why a given route was selected over another route. In the context of autonomous driving, such route planners may not be able to explain why an action (e.g., turn left, turn right, go straight, move to the next lane segment, etc.) is or was performed.

[0038] Some route planners may be implemented as neural networks (e.g., deep learning neural networks, etc.). Such route planners may not be able to provide human-understandable explanations. The inference model of such route planners will be weights and activation values ​​(e.g., millions of real numbers) that have no semantic meaning. The decisions (e.g., selected actions) of such route planners imply which nodes of the model are activated using weight values. Extracting explanations from such route planners, even if possible, may not be straightforward and may require additional systems that attempt to explain the route planner itself (e.g., the meaning of activation values ​​based on inputs). Thus, the route planner becomes an inexplicable black box. Activation values ​​that result in a particular action of the neural network may be output, but such activation values ​​are not easy to interpret and have no semantic meaning. In another example, to understand why an action was recommended by the route planner, it may be necessary to analyze the source code and the inputs to the source code to provide an explanation.

[0039] In contrast, lane-level route planners according to implementations of the present disclosure may be inherently explainable. A lane-level route planner may provide human-understandable explanations for why an action (equivalently, a route, a lane, or a lane segment) is selected. The explanations are human-interpretable and semantically meaningful. The explanations (e.g., sentences) may describe why an autonomous driving decision was made. For example, they may answer user questions such as "Why didn't you go right at the intersection?" or "Why are you asking me to take manual control?" Providing explanations for the selection of actions, roads, or lane segments in route planning may improve conventional route planners, particularly route planning for autonomous driving. Providing explanations may give users comfort in the decision-making of autonomous driving selections. Additionally, by understanding why a given selection is made, a user may influence and refine the route planning decision criteria to suit the user's preferences, for example, by providing different objectives and constraints on those objectives.

[0040] Additionally, because the lane-level route planner described herein can plan routes according to (e.g., using, based on) multiple objectives that have semantic meaning, the lane-level route planner can quantify or describe how good (or bad) a given route is for a given objective (traffic volume, preferences, human driving ability, automated driving, etc.).

[0041] In a further aspect of the present disclosure, a route can be planned using standard definition map data (or simply SD maps), and roads in the SD maps can be mapped to obtain HD map information.

[0042] A route planner typically obtains (e.g., calculates) a route using map data. In a conventional route planner, the map data may be SD map data. SD maps typically include roads, intersections, etc., but do not include sufficient detail or precision required for route planning (e.g., lane-level route planning) for autonomous driving (e.g., autonomous driving decision-making) and automated driving. SD maps may include coarse GPS coordinates for navigation (e.g., as opposed to automated driving decision-making). Autonomous driving may require high-definition (HD) maps.

[0043] HD map data may include information about vehicular traffic networks accurate to within a few centimeters. HD map data may include details about curbs, lane boundaries, crosswalks, traffic lights, road dividers, traffic signals, traffic signs, speed limits, landmarks, and more.

[0044] HD maps may be expensive and may not include information about a given road (or lanes on a road). Roads or areas may be said to be unmapped. HD maps do not include information about unmapped roads and areas.

[0045] Although an AV includes sensors that can detect HD information for decision-making purposes or trajectory planning, the AV may only include an SD map for navigation purposes. The SD map cannot be localized and matched by default to the HD information observed by the AV's sensors so that it can also be used for decision-making in autonomous driving. Thus, the problem is that the AV (i.e., modules therein) still needs to know the HD information for autonomous driving. According to the present disclosure, autonomous driving can be performed on roads with SD maps to obtain HD information for subsequent route planning.

[0046] In an aspect of the present disclosure, the SD map can be supplemented with information (e.g., HD information) using sensor data from one or more vehicles so that the SD map can be used for lane-level route planning described herein for autonomous driving. In this manner, supplementing the HD map with sensor data allows the lane-level route planner to obtain a route using the SD map or a combination of the HD map and the SD map. The HD map may be used for roads and lanes that have been mapped, and the SD map can be used for roads and lanes that have not been mapped to the HD map but for which HD information has been obtained as described herein. For ease of reference, as used herein, an augmented HD map refers to an HD map supplemented with data obtained using one or more AV's sensors for roads in the SD map. Additionally, or equivalently, the navigation map can be augmented with information for roads in the HD map as these roads are traversed.

[0047] The augmented HD map has many advantages, including eliminating the need for an HD map for route planning or decision making. For example, an AV or fleet of AVs can generate its own HD map as it travels along roads in an SD map. As described below, the generation of the HD map can occur via a planning belief-based route planner. Advantages also include that the HD map is adaptable. For example, if road lanes are reconfigured in the SD map (such as lane closures or new lanes added due to construction), the HD map can adapt to the changes.

[0048] The details of lane-level route planning are described herein first with reference to the environment in which it may be implemented.

[0049] FIG. 1 illustrates an example of a vehicle in which aspects, features, and elements disclosed herein may be implemented. In the illustrated embodiment, the vehicle 100 includes various vehicle systems. The vehicle systems include a chassis 110, a powertrain 120, a controller 130, and wheels 140. Additional or different combinations of vehicle systems may be used. For simplicity, the vehicle 100 is shown to include four wheels 140, but any other propulsion device, such as one or more propellers or treads, may be used. In FIG. 1, lines interconnecting elements, such as the powertrain 120, the controller 130, and the wheels 140, indicate that information, such as data or control signals, forces, such as power or torque, or both information and power, may be transferred between each element. For example, the controller 130 may receive power from the powertrain 120 and communicate with the powertrain 120, the wheels 140, or both, to control the vehicle 100, which may include accelerating, decelerating, steering, or otherwise controlling the vehicle 100.

[0050] 1 includes a power source 121, a transmission 122, a steering device 123, and an actuator 124. Any other element or combination of elements of the powertrain may be included, such as a suspension, a drive shaft, an axle, or an exhaust system. Although shown separately, wheels 140 may be included in the powertrain 120.

[0051] The power source 121 includes an engine, a battery, or a combination thereof. The power source 121 may be any device or combination of devices that operates to provide energy, such as electrical energy, thermal energy, or kinetic energy. In one example, the power source 121 includes an engine, such as an internal combustion engine, an electric motor, or a combination of an internal combustion engine and an electric motor, that operates to provide kinetic energy as motive power to one or more of the wheels 140. Alternatively or additionally, the power source 121 may include a potential energy device, such as one or more dry batteries, such as nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), solar cells, fuel cells, or any other device capable of providing energy.

[0052] The transmission 122 receives energy, such as kinetic energy, from the power source 121 and sends the energy to the wheels 140 to provide motive power. The transmission 122 may be controlled by the controller 130, the actuator 124, or both. The steering device 123 may be controlled by the controller 130, the actuator 124, or both and control the wheels 140 to steer the vehicle. The actuator 124 may receive signals from the controller 130 and may operate or control the power source 121, the transmission 122, the steering device 123, or any combination thereof to operate the vehicle 100.

[0053] In the illustrated embodiment, the controller 130 includes a positioning device 131, an electronic communication device 132, a processor 133, a memory 134, a user interface 135, a sensor 136, and an electronic communication interface 137. A smaller number of these elements may be present as part of the controller 130. Although shown as a single device, any one or more elements of the controller 130 may be incorporated into any number of separate physical devices. For example, the user interface 135 and the processor 133 may be incorporated into a first physical device and the memory 134 may be incorporated into a second physical device. Although not shown in FIG. 1, the controller 130 may include a power source 1210, such as a battery. Although shown as separate elements, the positioning device 131, the electronic communication device 132, the processor 133, the memory 134, the user interface 135, the sensor 136, the electronic communication interface 137, or any combination thereof, may be incorporated into one or more electronic devices, circuits, or chips.

[0054] The processor 133 may include any device or combination of devices now existing or later developed capable of manipulating or processing signals or other information, including optical processors, quantum processors, molecular processors, or combinations thereof. For example, the processor 133 may include one or more special purpose processors, one or more digital signal processors, one or more microprocessors, one or more controllers, one or more microcontrollers, one or more integrated circuits, one or more application specific integrated circuits, one or more field programmable gate arrays, one or more programmable logic arrays, one or more programmable logic controllers, one or more state machines, or any combination thereof. The processor 133 may be operatively coupled to one or more of the positioning device 131, the memory 134, the electronic communication interface 137, the electronic communication device 132, the user interface 135, the sensor 1360, and the powertrain 136. For example, the processor may be operatively coupled to the memory 134 via a communication bus 138.

[0055] Memory 134 includes any tangible, non-transitory computer usable or computer readable storage medium capable of, for example, holding, storing, transmitting or conveying machine-readable instructions or any information associated therewith for use by or in connection with processor 133. Memory 134 may be, for example, one or more solid-state drives, one or more memory cards, one or more removable media, one or more read-only memories, one or more random access memories, one or more disks, including hard disks, floppy disks, optical disks, magnetic or optical cards, or any type of non-transitory medium suitable for storing electronic information, or any combination thereof. For example, memory may be one or more read-only memories (ROM), one or more random access memories (RAM), one or more registers, low power DDR (LPDDR) memories, one or more cache memories, one or more solid-state memory devices, one or more magnetic media, one or more optical media, one or more magneto-optical media, or any combination thereof.

[0056] The communications interface 137 may be a wireless antenna as shown, a wired communications port, an optical communications port, or any other wired or wireless device capable of interfacing with a wired or wireless electronic communications medium 150. Although Figure 1 illustrates communications interface 137 communicating over a single communications link, the communications interface may be configured to communicate over multiple communications links. Although Figure 1 illustrates a single communications interface 137, the vehicle may include any number of communications interfaces.

[0057] The communication device 132 is configured to transmit or receive signals over a wired or wireless electronic communication medium 150, such as via a communication interface 137. Although not explicitly shown in FIG. 1, the communication device 132 may be configured to transmit, receive, or both over any wired or wireless communication medium, such as radio frequency (RF), ultraviolet (UV), visible light, optical fiber, wired line, or a combination thereof. Although FIG. 1 shows a single communication device 132 and a single communication interface 137, any number of communication devices and any number of communication interfaces may be used. In some embodiments, the communication device 132 includes a Dedicated Short Range Communications (DSRC) device, an On-Board Unit (OBU), or a combination thereof.

[0058] Positioning device 131 may determine geographic information such as the longitude, latitude, altitude, heading, or speed of vehicle 100. In one example, the GPS includes a Global Positioning System (GPS) device such as a Wide Area Augmentation System (WAAS)-enabled National Marine Electronics Association (NMEA) device, a radio triangulation device, or a combination thereof. Positioning device 131 may be used to obtain information representing, for example, the current heading of vehicle 100, the current location of vehicle 100 in two or three dimensions, the current angular heading of vehicle 100, or a combination thereof.

[0059] The user interface 135 includes any device capable of interfacing with a person, such as a virtual or physical keypad, touch pad, display, touch display, heads-up display, virtual display, augmented reality display, tactile display, feature tracking device such as an eye-tracking device, speaker, microphone, video camera, sensor, printer, or any combination thereof. The user interface 135 may be operatively coupled to the processor 133 as shown, or to any other element of the controller 130. Although shown as a single device, the user interface 135 may include one or more physical devices. For example, the user interface 135 may include both an audio interface for voice communication with the person, and a touch display for visual and touch-based communication with the person. The user interface 135 may include multiple displays, such as multiple physically separate devices, multiple defined portions of a single physical device, or a combination thereof.

[0060] The sensors 136 are operable to provide information that can be used to control the vehicle. The sensors 136 may be an array of sensors. The sensors 136 may provide information about the current operating characteristics of the vehicle 100, including vehicle operation information. The sensors 136 may include, for example, speed sensors, acceleration sensors, steering angle sensors, traction-related sensors, brake-related sensors, steering wheel position sensors, eye tracking sensors, seating position sensors, or any sensor or combination of sensors operable to report information about some aspect of the current dynamic situation of the vehicle 100.

[0061] Sensors 136 may include one or more sensors operable to obtain information about the physical environment surrounding vehicle 100, such as operating environment information. For example, one or more sensors may detect road geometry, such as lanes, and obstacles, such as fixed obstacles, vehicles, and pedestrians. Sensors 136 may be or include one or more video cameras, laser sensing systems, infrared sensing systems, acoustic sensing systems, or any other suitable type of on-board environment sensing device or combination of devices, now known or later developed. In some embodiments, sensors 136 and positioning device 131 are combined.

[0062] Although not separately shown, the vehicle 100 may include a trajectory controller. For example, the controller 130 may include a trajectory controller. The trajectory controller may be operable to obtain information describing the current state of the vehicle 100 and a planned route for the vehicle 100, and to determine and optimize a trajectory for the vehicle 100 based on this information. In some embodiments, the trajectory controller may output signals operable to control the vehicle 100 such that the vehicle 100 follows a trajectory determined by the trajectory controller. For example, the output of the trajectory controller may be an optimized trajectory that may be provided to the powertrain 120, the wheels 140, or both. In some embodiments, the optimized trajectory may be a control input, such as a set of steering angles, each steering angle corresponding to a time or position. In some embodiments, the optimized trajectory may be one or more paths, lines, curves, or combinations thereof.

[0063] One or more of the wheels 140 may be steering wheels that are pivoted to a steering angle under the control of the steering device 123, propulsion wheels that are given a torque to propel the vehicle 100 under the control of the transmission 122, or steering and propulsion wheels that may both steer and propel the vehicle 100.

[0064] Although not shown in FIG. 1, the vehicle may include additional devices or elements not shown in FIG. 1, such as an enclosure, a Bluetooth® module, a frequency modulation (FM) radio device, a near field communication (NFC) module, a liquid crystal display (LCD) display device, an organic light emitting diode (OLED) display device, a speaker, or any combination thereof.

[0065] Vehicle 100 may be an autonomous vehicle that is autonomously controlled without direct human intervention to navigate a portion of a transportation network. Although not separately shown in FIG. 1 , the autonomous vehicle may include an autonomous vehicle controller that provides routing, navigation, and control of the autonomous vehicle. The autonomous vehicle controller may be integrated with other devices of the vehicle. For example, controller 130 may include an autonomous vehicle controller.

[0066] If present, the autonomous vehicle controller may control or operate the vehicle 100 to move through a portion of the vehicle traffic network according to the current vehicle operating parameters. The autonomous vehicle controller may control or operate the vehicle 100 to perform a defined operation or maneuver, such as parking the vehicle. The autonomous vehicle controller may generate a travel route from a starting point, such as the current location of the vehicle 100, to a destination based on the vehicle information, the environmental information, the vehicle traffic network information representing the vehicle traffic network, or a combination thereof, and may control or operate the vehicle 100 to move through the vehicle traffic network according to the route. For example, the autonomous vehicle controller may output the travel route to a trajectory controller to operate the vehicle 100 to move from the starting point to the destination using the generated route.

[0067] Figure 2 is an example diagram of a portion of a vehicle traffic and communication system in which aspects, features and elements disclosed herein may be implemented. The vehicle traffic and communication system 200 may include one or more vehicles 210 / 211, such as the vehicle 100 shown in Figure 1, that may travel through one or more portions of a vehicle traffic network 220 and communicate through one or more electronic communication networks 230. Although not explicitly shown in Figure 2, the vehicles may travel through off-road areas.

[0068] Electronic communications network 230 may be a multiple access system providing communications, such as, for example, voice, data, video, messaging, or combinations thereof, between vehicles 210 / 211 and one or more communications devices 240. For example, vehicles 210 / 211 may receive information, such as information describing vehicular traffic network 220, from communications device 240 via network 230.

[0069] In some embodiments, vehicles 210 / 211 may communicate via a wired communication link (not shown), a wireless communication link 231 / 232 / 237, or any combination of any number of wired or wireless communication links. As shown, vehicles 211 / 211 communicate via a land-based wireless communication link 231, via a non-land-based wireless communication link 232, or via a combination thereof. Land-based wireless communication link 231 may include an Ethernet link, a serial link, a Bluetooth link, an infrared (IR) link, an ultraviolet (UV) link, or any link capable of providing electronic communications.

[0070] Vehicles 210 / 211 may communicate with other vehicles 210 / 211. For example, a host or target vehicle 210 may receive one or more autonomous vehicle-to-vehicle messages, such as a basic safety message (BSM), from a remote or target vehicle (RV) 211 via a direct communication link 237 or via a network 230. The remote vehicle 211 may broadcast a message to the host vehicle within a defined broadcast range, such as 300 meters. In some embodiments, the host vehicle 210 may receive the message via a third party, such as a signal repeater (not shown) or another remote vehicle (not shown). Vehicles 210 / 211 may transmit one or more autonomous vehicle-to-vehicle messages periodically based on a defined interval, such as 100 milliseconds.

[0071] The automatic vehicle-to-vehicle messages may include vehicle identification information, geospatial state information such as longitude, latitude or altitude information, geospatial position accuracy information, motion state information such as vehicle acceleration information, yaw rate information, speed information, vehicle heading information, braking system state information, throttle information, steering angle information or vehicle routing information, or vehicle operating state information such as vehicle size information, headlight state information, turn signal information, wiper state information, transmission information or any other information or combination of information related to the transmitting vehicle state. For example, the transmission state information may indicate whether the transmitting vehicle's transmission is in neutral, parked, forward or reverse.

[0072] The vehicle 210 may communicate with the communication network 230 via an access point 233. The access point 233, which may include a computing device, is configured to communicate with the vehicle 210, the communication network 230, one or more communication devices 240, or a combination thereof, via wireless or wired communication links 231 / 234. For example, the access point 233 may be a base station, a base transceiver station (BTS), a Node-B, an enhanced Node-B (eNode-B), a Home Node-B (HNode-B), a wireless router, a wired router, a hub, a relay, a switch, or any similar wired or wireless device. Although shown here as a single device, the access point may include any number of interconnection elements.

[0073] Vehicle 210 may communicate with communication network 230 via satellite 235 or other non-terrestrial communication devices. Satellite 235, which may include a computing device, is configured to communicate with vehicle 210, communication network 230, one or more communication devices 240, or combinations thereof, via one or more communication links 232 / 236. Though shown here as a single unit, the satellite may include any number of interconnecting elements.

[0074] Electronic communications network 230 is any type of network configured to provide voice, data, or any other type of electronic communications. For example, electronic communications network 230 may include a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a mobile or cellular telephone network, the Internet, or any other electronic communications system. Electronic communications network 230 uses communications protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Internet Protocol (IP), Real-time Transport Protocol (RTP), Hypertext Transport Protocol (HTTP), or combinations thereof. Although shown here as a single device, electronic communications network may include any number of interconnected elements.

[0075] The vehicle 210 may identify a portion or state of the vehicular traffic network 220. For example, the vehicle includes at least one on-board sensor 209, such as the sensor 136 shown in FIG. 1, which may be or include a speed sensor, a wheel speed sensor, a camera, a gyroscope, an optical sensor, a laser sensor, a radar sensor, an acoustic sensor, or any other sensor or device capable of determining or identifying a portion or state of the vehicular traffic network 220, or a combination thereof.

[0076] Vehicle 210 may navigate one or more portions of vehicular traffic network 220 using information communicated over network 230, such as information representative of traffic network 220, one or more onboard sensors 209, or a combination thereof.

[0077] For simplicity, one vehicular traffic network 220, one electronic communications network 230, and one communications device 240 are shown in Figure 2, although any number of networks or communications devices may be used. Vehicular traffic and communications system 200 may include devices, equipment, or elements not shown in Figure 2. Although vehicle 210 is shown as a single unit, the vehicle may include any number of interconnected elements.

[0078] Although vehicle 210 is shown communicating with communication device 240 via network 230, vehicle 210 may communicate with communication device 240 via any number of direct or indirect communication links. For example, vehicle 210 may communicate with communication device 240 via a direct communication link, such as a Bluetooth® communication link.

[0079] 3 is a flow chart diagram illustrating an example of a technique for route planning according to an embodiment of the present disclosure. The technique 300 may be implemented partially or fully in a host vehicle, which may be the vehicle 100 shown in FIG. 1, one of the vehicles 210 / 211 shown in FIG. 2, a semi-autonomous vehicle, any other vehicle that may include driver assistance features, or a vehicle that is manually controlled by a driver or the like. The technique 300 may be implemented as instructions (e.g., operations) stored in a memory, such as the memory 134 of FIG. 1. The instructions may be executed by a processor, such as the processor 133 of FIG. 1. The technique 300 may be implemented using dedicated hardware or firmware. Multiple processors, memories, or both may be used.

[0080] The technique 300 may be implemented partially or completely by a lane level route planner. The lane level planner may be implemented using a processor including a CPU (Central Processing Unit), a memory, and an input / output device as described with reference to FIG. 1 and the like. A computer program that causes the processor to function as the lane level route planner may be installed and executed on the processor. This allows the processor to function as the lane level route planner. Note that, although an example in which the lane level route planner is implemented by software will be described here, it goes without saying that the lane level route planner may also be implemented by dedicated hardware prepared to execute each information processing described here.

[0081] In step 302, the technology 300 receives a destination. In one example, a user (e.g., a driver) can provide the destination via a user interface of a routing application. In one example, a user can provide the destination to a routing application that may be running on the user's portable device. In another example, a vehicle is running a service (e.g., a robotaxi service) and the destination can be provided to the technology 300 via a dispatch process, whereby the vehicle travels to the destination, which may be a customer or baggage pick-up location or a customer or baggage drop-off location. Other ways of providing the destination to the technology 300 are also possible. The destination may be an address, a landmark, a venue, or other destination type. The technology 300 can convert the received destination into a set of GPS coordinates.

[0082] In step 304, the technique 300 obtains a lane level route to the destination using a map (i.e., a navigation map or a lane level navigation map). The lane level route includes a transition (i.e., an action for the transition) from a first segment of a first lane of a road to a second segment of a second lane of the road. In one example, the destination may be transmitted (wirelessly) and provided to a server (e.g., a cloud-based server), such as the communication device 240 of FIG. 2, which calculates the lane level route. An example of a map and lane level route is described with respect to FIG. 4.

[0083] 4 is a diagram illustrating an example of a map according to an implementation of the present disclosure. Map 400 shows that a destination is on road segment 401. Map 400 includes road 402 that includes three lanes, lanes 404A-404C. The lanes in map 400 are divided into lane segments. The lane segments are indicated by numbered segments in map 400. For example, lane 404A includes lane segments 406A-406C. Vehicle 403 is currently on lane segment 406A and is heading toward the destination of lane segment 401. Lane segment 406A may be an intermediate lane segment between a starting point (not shown) of vehicle 403 and the destination.

[0084] In one example, all road segments may have the same length (e.g., 100 meters, 200 meters, or other segment length). In one example, the length of a lane segment may depend at least in part on the speed along the road of the lane segment. For example, at high speeds, the lane segment may have a longer length, such as 500 meters. In one example, the length of some lane segments may be adjusted. That is, for example, when generating a first lane level route, the first lane level route may be generated by a portion of the lane segments having a first length, and when generating a second lane level route, the second lane level route may be generated by a portion of the lane segments having a second length. The length of the lane segments may be changed based on the speed on the road, the time of day (e.g., rush hour vs. weekend), socio-geographic region (e.g., school zone, residential area), weather (e.g., sunny vs. snow), road type (e.g., highway vs. city road), more, less, other criteria, or combinations thereof. For example, as visually illustrated, lane segment 406E is shorter than lane segment 406C, but lane segment 406D is longer than lane segment 406C.

[0085] In map 400, lane segments may have identification numbers (IDs). For example, lane segments 406A-406E have IDs 1, 2, 3, 7, and 13, respectively. At least some of the lane segments may be associated with metadata. In one example, the metadata may include one or more classifications (e.g., designations). For example, map 400 may indicate (using different shading) that lane segments with IDs 1, 2, and 3 are high-speed segments, lane segments with IDs 4, 5, 6, 7, and 11 are standard-speed (e.g., speed-limited) segments, lane segments with IDs 8, 9, and 10 are comfort (e.g., low-speed) segments, lane segments with IDs 12, 13, and 15 are city lane segments, and lane segments with IDs 14, 16, and 17 are manual-driving lane segments. These classifications are not necessarily mutually exclusive. For example, an urban lane segment may be a manually driven lane segment.

[0086] A comfort lane segment may be a lane segment along which traffic may travel at a speed within -10 miles per hour (or other threshold speed) of the speed limit for the roadway of the lane segment. A manually driven lane segment is a lane segment that cannot be traversed autonomously or cannot be traversed autonomously with sufficient confidence. Such lane segments must be traversed under manual driving control, such as by a driver or teleoperator. As such, the lane-level route planner may include an action to notify the user (e.g., the driver) a certain time prior to the lane segment that the user should take over manual driving control of the vehicle.

[0087] In one example, obtaining a lane level route may include obtaining a policy for traversing lane segments from a starting location to a destination. The policy may include lane transitions. More precisely, as described below, the policy provides actions given that the vehicle is in a current lane segment. In one example, the policy may be a deterministic policy. In another example, the policy may be based on a probabilistic model.

[0088] In route planning, minimizing the time to a destination is the typical objective of a route planner, but the lane-level route planner according to the present disclosure can obtain a lane-level route by optimizing for several objectives, as further described below.

[0089] Lane-level routes can be obtained using a type of Markov decision process (MDP) model. Lane-level route planning can be modeled as a topological partially observable Markov decision process (TPOMDP) ​​or its fully observable subclass, the topological Markov decision process (TMDP). TPOMDP and TMDP allow for incorporating additional objective measures, such as maximizing safety, smoothness, and / or other people's preferences, into the objectives of a typical partially observable Markov decision process model (POMDP) ​​or Markov decision process (MDP), such as minimizing travel time or travel distance. Solving a TPOMDP model can be similar to solving a TMDP model, except that TPOMDP is based on belief states (i.e., probabilities), which represent the probability of each state, followed by the observation probability corresponding to the generation of the observations of each state. Other model types are possible. For example, the model can be a scalarized multiobjective Markov decision process (MOMDP), a scalarized partially observable multiobjective Markov decision process (MOPOMDP), a constrained Markov decision process (CMDP), or a constrained partially observable Markov decision process (CPOMDP).

[0090] To illustrate, without loss of generality, a user may indicate a preference for slow lanes, so that the route planner can incorporate a "comfort" objective into the calculation of how to reach the destination. In another example, a user may additionally or alternatively indicate a preference for lanes that minimize energy consumption. For example, if traffic in some lane segments tends to be slow and require more energy consumption, the route planner may prioritize other road segments. As another example, smooth roads are prioritized by the route planner over roads with many uphill and downhill slopes, since such roads tend to require more energy consumption. As another example, a road with many charging stations may be prioritized over another road with fewer charging stations.

[0091] Without loss of generality, the model (eg, TMDP or TPOMDP) ​​can be described in stochastic shortest path (SSP) structural terms (eg, costs instead of rewards, and goals as target states that self-loop upon arrival).

[0092] A connected graph (V, E, R) is a tuple <S,A,T,C,E,δ,s0,s g >. Each vertex v∈V may be the GPS coordinate of a point where a routing decision may be made. Thus, a vertex may be the GPS coordinate of an intersection, a lane change location (e.g., from lane segment ID9 to lane segment ID11), a junction, etc. To illustrate, in map 400, a vertex v may be the coordinate of a particular point of a lane segment. For example, the particular point may be the midpoint of a lane segment. Each edge e∈E of the connectivity graph may describe a directed graph of vertices of length or travel time R(e). For example, in the connectivity graph below map 400, the vertex corresponding to lane segment ID9 is connected to lane segments IDs 10 and 11.

[0093] Next, we outline the formal model TMDP. ​​As mentioned before, this model is formally <S,A,T,C,E,δ,s0,s g> S. S may be a finite set of states or vertices v∈V. A is the action space at each vertex for selecting a subsequent lane segment (in other words, a subsequent vertex). To illustrate without loss of generality, the action space A may include the actions "move left", "move right", "move straight", or "move diagonally". However, more, less, other actions, or combinations thereof are possible. For example, the action space A may include actions to inform the user to take over manual control, to switch to manual control, to inform the user that control is switching to automatic control, to switch to automatic control, etc.

[0094] s0 is the initial state, which may be the current position of the vehicle when the route planner calculates the lane-level route. g is the goal state, which can be the destination. g When the goal state is reached, no matter what action is taken, the goal state is maintained, and the goal state s g All actions performed with have a cost of 0.

[0095] T (i.e., T: S×A×S→[0,1]) may be a state transition function that describes the probability of a subsequent state s′∈S occurring after performing an action a∈A in state s∈S. Thus, T is a state transition function that describes the probabilistic success / failure of each action. The state transition function T(s,a,s′)=Pr(s′|s,a) is the probability s′ of a subsequent (lane segment or vertex) given that action a was performed in state s. In one example, the state transition function may include respective probabilities of at least some (e.g., each) of the adjacent vertices (e.g., lane segments) of a vertex (e.g., lane segment). To illustrate, for a lane segment ID1 (i.e., lane segment 406A), the state transition function T may include respective transition probabilities for some of the transitions from the lane segment ID1 to the lane segment ID4, the transitions from the lane segment ID1 to the lane segment ID2, and / or the transitions from the lane segment ID1 to the lane segment ID5. In some examples, the state transition function may include a transition probability to a more distant neighbor (eg, from lane segment ID1 to lane segment ID3).

[0096] As such, using the map to obtain a lane-level route to the destination may include obtaining a policy that provides an action to control the AV, when the AV is on a first lane segment of the map, to move into a second lane segment adjacent to the first lane segment.

[0097] The probabilities may be derived based on criteria including speed limits, lane lengths of edge e, AV specifications, time and location, traffic congestion information, more criteria, fewer criteria, other criteria, or combinations thereof. In one example, these probabilities (and / or the criteria used to derive the transition probabilities) may be learned, as described further below.

[0098]

number

number

[0099] The objectives may be arranged in a preference ranking that follows a directed acyclic graph (DAG) E. An example preference ranking graph is illustrated in Figure 5. Each objective may have a non-negative slack δ, where δ :e → R+ describes how much a user is willing to "spend" on the value of one objective to improve the value of another objective. Thus, one or more objectives are related in a preference ranking that includes a slack variable.

[0100] A solution to the model may be a policy π:S→A. Under policy π, an action a (i.e., π(s)) is selected for state s. That is, policy π may indicate that action π(s)∈A should be executed at state s. Policy π is a method for moving from state s to a destination (i.e., goal state s) according to policy π. g ) the expected cumulative cost to reach π A value function V that can represent (s) π : S → C. That is, the value function is calculated for each intermediate state s from the starting state to the goal state. i , may provide the expected cost (i.e., value) of

[0101] Optimal policy π * minimizes the expected cumulative cost. Formally, for the initial state s0, the expectation can be given by equation (1).

number

[0102] In formula (1), S t and A t Let π denote the state and action random variables at time step t, respectively. In general, to solve a model, an appropriate policy must exist. An appropriate policy π has the following properties: (1) there exists a policy that reaches the goal with probability 1, and (2) every state that does not reach the goal with probability 1 incurs infinite cost.

[0103] In step 306 of FIG. 3, the technique 300 controls the AV to travel the lane-level route. Returning to FIG. 4, the arrows in FIG. 4 indicate actions under the determined policy π. Arrow 408A indicates the action "if the state is equal to the lane segment of ID1, perform the right action, so that the vehicle ends up in a state equal to the lane segment of ID4." In this case, "perform the action" means that "the vehicle is controlled" by changing the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle of the steering wheel, etc.

[0104] The determined policy π incorporates (e.g., includes) a contingency route. Thus, the resulting lane-level route includes the contingency route. To illustrate, if the vehicle is in lane segment ID9, the policy indicates that the vehicle is controlled to move to lane segment ID11. However, if the vehicle is unable to take lane segment ID11 and enters lane segment ID10, the policy indicates "perform left action" to move the vehicle to lane segment ID6 and then to lane segment ID7. Thus, controlling the AV to traverse the lane-level route may include controlling the AV according to the contingency route in response to failing to transition from the first segment to the second segment.

[0105] FIG. 5 illustrates an example preference ranking graph 500 of objectives according to an implementation of the present disclosure. As described above, the objectives of the model may be topologically ordered. The topologically ordered constraints may satisfy the slack of the first belief or all beliefs against the preceding objectives. A lane-level route planner can reason about the objectives to compute an ordered policy in the topologically ordered constraint graph. Although three different preference rankings are described, the present disclosure is not limited thereto and other configurations (i.e., preference rankings) are possible.

[0106] The objectives shown in example 500 relate to speed optimization (i.e., bubbles labeled H, such as objectives 512, 538, and 556), comfort optimization (i.e., bubbles labeled C, such as objectives 514, 534, and 554), city driving optimization (i.e., bubbles labeled U, such as objectives 516, 532, and 552), and manual driving optimization (i.e., bubbles labeled M, such as objectives 518, 536, and 558). The illustrated objectives are examples, and other objectives are possible. Furthermore, the illustrated preference ranking graph is a non-limiting example, and other preference ranking graphs including the same or different objectives and topologies are possible. Each of these objectives is associated with a respective semantic label that may be meaningful to a human. As described further herein, a user may use the semantic labels to indicate route preferences for a lane-level route planner.

[0107] Example 510 is an example of a chained preference ranking graph. When using the preference ranking graph of example 510 (i.e., when the objectives are linked in a chain), first the high speed objective is maximized (i.e., objective 512), then the comfort objective (i.e., objective 514) is maximized if it is within the high speed objective's slack 513 (e.g., a slack of 3), then the city objective (i.e., objective 516) is maximized if it is within the comfort objective's slack 515 (e.g., a slack of 1), which has already been constrained to ensure that it is within the first objective's slack 513, and finally the manual driving objective (i.e., objective 518) is maximized if it is within the already constrained city objective's slack 517 (e.g., a slack of 7).

[0108] Example 510 may be interpreted (e.g., understood) as a lane-level route planner preferring the highway (i.e., objective 512), but allowing 3 minutes of extra time (i.e., slack 513) in order for the route planner to select another alternative route that is more comfortable (i.e., objective 514), which also maximizes the comfort objective. So, a user may provide input (e.g., issue a command) that essentially indicates, "Give me the fastest road possible, but allow me a route that is 3 minutes longer than the fastest in order to drive in the more comfortable lane," and similarly for the other objectives in example 510. It should be noted that the unit of measure of the slack variable is the same as the unit of measure of the objective it constrains.

[0109] As described further below, the user can specify desired preferences using verbal commands that the route planner can convert into a preference ranking graph via a graphical user interface (e.g., a drag-and-drop interface, a list, etc.) or via other manners for the user to provide the preferences.

[0110] Example 530 is an example of a sector-shaped preference ranking graph. When the preference ranking graph of example 530 is used (i.e., the objectives have such a sector-like relationship), optimal policies are calculated simultaneously for the city objective (i.e., objective 532), the comfort objective (i.e., objective 534), and the manual driving objective (i.e., objective 536). Then, a final policy is determined that maximizes the high speed objective (i.e., objective 538) subject to the slacks 533, 535, and 537 (i.e., slack values ​​5, 9, and 2, respectively) of the three initial objectives. This is equivalent to taking the intersection of the policy sets of the first three objectives and searching that space for the final objective.

[0111] Example 550 is an example of a directed graph that mixes examples 510 and 530.

[0112] For each objective (e.g., a bubble in example 500), the lane-level route planner obtains a set of routes that satisfy the objective. For example, for objective 552, the lane-level route planner identifies an optimal route (e.g., a minimal route to reach the destination with as many city lanes as possible). A slack of one minute (i.e., slack 553) is allowed to reach the goal. Thus, the route planner can be thought of as providing an envelope of possibilities around reaching the goal. That is, the route planner can identify all routes that are within one minute of the most optimal time, and whether they are on city roads or not. Thus, an initial set of routes that satisfy objective 552 within slack 553 is returned.

[0113] A second set of routes meets objective 554 (i.e., stay in lanes marked as comfortable) and is within 6 minutes (i.e., slack 555) of the most comfortable route. Similarly, a third set of routes is obtained that meets the manual driving objective within 9 minutes (i.e., slack 557), and so on. The full set of retrieved routes is passed to objective 556, and routes retrieved from this sink objective must be in all of the sets of retrieved routes. In some situations, a route that meets all constraints and slack may not be possible.

[0114] Note that the incoming arrows to the bubbles in example 500 can be thought of as constraining that objective to be within a given set of routes (or sets, depending on the configuration of the preference ranking graph). Thus, in the absence of a parent arrow (i.e., no incoming arrows), any route that satisfies the objective within any slack (if any) may be selected by the route planner. On the other hand, the selected route(s) must be within the routes provided by the parent objective. Thus, downstream objectives can be thought of as pruning the set(s) of provided (e.g., incoming) routes.

[0115] In one example, instead of a preference ranking graph, a scalarization function can be used to map (e.g., combine, etc.) all rewards (e.g., costs) to a single value. The scalarization function can be used to convert the model / problem into a shortest path optimization problem (SSP) MDP or POMDP. Using the scalarization function, a single value indicating the long-term utility of the next immediate action may be obtained, which combines the expected costs to obtain the single value. Formally, the scalarization function f:C→R is expressed as f(V(s))=V in terms of the single value. f (s), and in terms of cost function f(C(s))=C f (s). In one example, the scalarization function may be a weighted sum of the objectives or a non-linear function of the objectives. Different weights may be used depending on the desired optimization. In one example, lane-level route planning may be modeled as a classical planning (CP) problem by eliminating probabilities.

[0116] Regardless of the model used, lane level route planning may be hierarchical. As can be appreciated, the greater the number of possible lanes, if any, the longer the policy may take to compute. To limit the number of lane segments that the lane level route planner infers, lane level route planning may be performed in a hierarchical manner. For example, for an origin in Miami, Florida, and a destination in Fairbanks, Alaska, the lane level route planner may group US roads into clusters, plan lane level routes between the clusters, and then perform route level planning within each cluster of the route.

[0117] In one example, as discussed above, lane-level route planning may be modeled as a topological partially observed MDP (TPOMDP) ​​and allow for beliefs about, for example, (1) traffic levels in each state, (2) the capabilities of at least one of the drivers and / or vehicles, and / or (3) various objective costs.

[0118] More formally, TPOMDP is a tuple<S,A,Ω,T,O,R,E,δ> It can be written as:

[0119] S is a finite set of states. A is a finite set of actions. Ω is a finite set of observations. T is the state transition function, where T(s,a,s')=Pr(s'|s,a) is the probability of subsequent state s' given that action a was executed in state s. O is the observation function, where O(a,s',ω)=Pr(w|a,s') is the observation probability ω given that action a was executed and the result was subsequent state s'.

[0120] R is a vector of reward functions, which may be equivalent to the cost vector C discussed above. As discussed above, E is a set of edges for k rewards (e.g., costs), forming a directed acyclic graph with one leaf (i.e., sink) reward vertex, which is reward vertex k, without loss of generality. δ is the sum of the edges e =<i,j> ∈E to non-negative slack constraints δ(e)≧0 or equivalently δ(i,j)≧0.

[0121] TPOMDP is a set of world beliefs b∈B⊆Δ |S| where Δ |S| is a probability distribution over state S, and as a standard |S|-simplex. Belief b is a probability distribution over states. Beliefs can exist at all time steps and for all forward, backward, following, leading, etc. lane segments. For belief b, after taking action a and making observation ω, the next belief b for state s' is ba ω is

number

[0122] As mentioned above, TMDP is a subclass of TPOMDP with Ω=S and O(a,s,s')=1, and an attainable belief b∈B is b(s)=1 for all s∈S.

[0123] The objective of the infinite horizon TPOMDP is to maximize the discounted reward expected from an initial belief b0 with a discount factor γ∈[0,1). For a policy π, the expected discounted reward is

number

[0124] Value V π :B→R is the expected reward at belief b and can be given by:

number

[0125] In equation (3), R(b,a)=Σ s b(s)R(s,a) and b b π(b)ω' follows the belief update in equation (2).

[0126] As described herein, the lane level route planner derives the policies using a navigation map. In one example, the navigation map may be learned. As used herein, learned includes that the navigation map may be evolved. The navigation map may include pre-populated lane information, and as lane level information is obtained from one or more vehicles, the pre-populated lane information may be evolved to reflect (e.g., incorporate) the obtained lane level information. The navigation map may not initially include lane level information, and the lane level information may be built (e.g., populated) based on lane level information received from one or more vehicles.

[0127] The lane level information may be received continuously and the navigation map may be periodically updated using newly received lane level information. The navigation map may be a vehicle specific navigation map (e.g., deployed in a particular vehicle), and the navigation map may be updated based on lane level information from the vehicle itself.

[0128] Additionally or alternatively, the navigation map may be learned based on lane-level information received (such as at a server) from many vehicles. The lane-level information may be with the server, which may be a cloud-based server, so that multiple vehicles may then receive accurate initial estimates for time (before actually driving on the road), preferences for popular routes (in new areas), and capability information for where the fleet of vehicles (of vehicles sharing the same code / hardware) should be planning to travel along the route.

[0129] Lane level information may be obtained from many vehicles. Vehicles owned by a particular customer may generate lane level information. A deployed fleet of vehicles (such as those providing taxi services, delivery services, and other services) may generate lane level information. Lane level information from a vehicle may be generated using data received from sensors on the vehicle. In one example, raw sensor data may be sent to a server, and the server may generate lane level information using the raw sensor data.

[0130] In this manner, the navigation map may include learned historical driving patterns and characteristics for different lanes and different lane segments of different roads. The historical driving patterns may be either for a particular vehicle, a particular driver of a particular vehicle, or the aggregated learned historical driving patterns of multiple vehicles.

[0131] The driving patterns of one or more drivers may be learned and used for lane-level route planning. The navigation map may include information about road conditions. Thus, the learned navigation map may leverage / incorporate the driver's driving behavior history and / or characteristics of lanes (more specifically, lane segments) to enable lane-level route planning.

[0132] Lane-level information from multiple connected vehicles (e.g., connected to a cloud-based server) may be combined into a shared navigation map. Each connected vehicle may send lane-level information to a server, such as one or more communication devices 240, which can aggregate all of the obtained lane-level information to obtain a navigation map. Multi-vehicle multi-objective learning (i.e., lane-level information from multiple connected vehicles) can significantly speed up the learning of a single vehicle, allowing the vehicle to learn route values ​​(e.g., lane-level information) before driving on the road itself.

[0133] Over time, the server may receive information regarding many lane segments that many drivers have traveled in the area, as well as roads that a particular vehicle has traveled on. The lane level information may also include an identification of the vehicle platform (e.g., Nissan Leaf, Nissan Sentra, Nissan Rogue, etc.). In some examples, the lane level information may also include information (e.g., preferences, etc.) of the particular driver for whom the lane level information was generated. Using the received lane level information trace, the server can separate the information into those that are dependent on the vehicle (i.e., vehicle specifications), driving style, road (e.g., including lane and lane segment conditions), and / or other criteria to create unique navigation maps, each specific to a particular vehicle, a particular vehicle platform, a particular driver, a particular driving style, etc.

[0134] As described above, regarding the lane segments of the lanes of the road, and through learning, at least some environmental information, vehicle information, human information, lesser, more, other information, or combinations thereof may be learned. The learned information may be converted into probabilities. The probabilities may be incorporated into or used by state transition functions, as described herein. The probabilities may also be used in setting objectives regarding the route (e.g., policy) to be calculated.

[0135] The environmental information may include one or more of traffic patterns, pedestrian patterns, lane segment impassability information by vehicles (e.g., other AVs), weather information, load information, more, less, other environmental information, or a combination thereof. The vehicle information may include one or more of average speed information per lane segment, energy use (e.g., battery use for electric vehicles, gasoline and battery use for hybrid vehicles, or gasoline use for internal combustion engine vehicles), more, less, other vehicle information, or a combination thereof.

[0136] The human information may include one or more of a preferred route, a preferred lane, a request for control transfer (such as from automated to manual or from manual to automated), a capability model update with driver override, more, less, other human information, or a combination thereof. The request for control transfer may include the lane segment for which control transfer is requested. The request for control transfer may include the planned subsequent lane segment and which subsequent lane segment the human driver has taken.

[0137] As the vehicle is traversing a route (whether based on the current state of a navigation map or not setting the route via a lane level planner), the vehicle's sensors can be used to gather information that is converted into lane level information, i.e. the vehicle can be thought of as being in shadow mode and monitoring the environment via its sensors.

[0138] Data from the sensors may be used to identify, for example, without limitation, whether a road (and thus lanes and lane segments) is urban or freeway, whether a lane segment is difficult to navigate, whether traffic is congested, the level of density, etc. Difficulty may be identified based on the amount of driver override, i.e., the number of times an automated driving decision is overridden or modified by a human, or the number of times a human instructs the automated control to control the vehicle in a different manner than suggested by the automated control.

[0139] The load information may indicate the level of stress that a user (e.g., a vehicle occupant) may be under given a selected lane change decision. To illustrate, a particular route may require the vehicle to move three lane segments to the left in a relatively short time or distance in a congested or fast traffic area. Such a situation may be stressful for the user. Such load information may be used to select a lane segment transition point much earlier than the congested area, or to select a route that avoids the stressful situation altogether.

[0140] Using lane level information, counts may be performed such that the counts can be converted to probabilities. For example, without loss of generality, a count of traffic density may be converted to a probability of moving from one lane segment to another, such as the time it takes to travel one lane segment of a road to move to the next, a count of the historical number of times traffic was light and heavy on a lane segment may be used to determine a probability of traffic density, a count of the number of times speeds on a lane segment may be used to classify the lane segment as being high speed, a count of the number of times speeds on a lane segment were within -10 miles of the speed limit may be used to determine the probability of the lane segment being a comfort lane segment, and so on.

[0141] The navigation map may be constructed from probabilities, i.e., probabilities may be incorporated into the navigation map. The navigation map is a transition graph as described with respect to FIG. 4. In deriving the policy, the next action (e.g., move straight to the next lane segment, move to the next lane segment to the left, etc.) may be determined based on lane level information (e.g., probabilities). To illustrate without loss of generality, the next action may be determined probabilistically (or deterministically) based on the probabilities of adjacent lane segments described herein, including, for example, the probability that the adjacent lane segment is congested based on whether the current lane segment is congested.

[0142] At least a portion of the lane-level information of the navigation map may be used to set lane-level route planner objectives. For purposes of illustration, and not limitation, as discussed above, the environment may include difficult passages, and if the objective (e.g., set by the user) is to maximize automated driving, the route planner will avoid lanes (or lane segments) that are difficult to pass. That is, the route planner policies are calculated to avoid lane segments that are difficult to pass.

[0143] Consider objectives regarding time to destination (which may include or incorporate speed limits, road length, stop signals, traffic density, whether the lane segment is a freeway, etc.), preferences (which may include at least some user preferences such as comfort, city roads as opposed to highways, energy consumption, etc.), and capabilities (which may include whether the lane segment can be traversed automatically or manually).

[0144] For illustration, but not by way of limitation, whenever a vehicle travels a route, the time to automatically traverse a particular lane segment may be recorded with or without a full stop, such that a route may be planned using at least some of the time, priority, and capacity objectives. Similarly, the time to manually traverse a particular lane segment (i.e., when a human is driving) may also be recorded. Thus, four average durations may be recorded. The average time may be used to define the TMDP state transition for the self-loop (stop). The average time may be used to define the reward / cost of traversal time, for example, based on the state of automated driving (e.g., control) versus manual driving.

[0145] Additionally, whenever a human drives a lane segment, subsequent lane segments may be recorded. The subsequent lane segments may be used to define rewards / costs for preferences in the range [0, 1], which may be the proportion of time the human drives a road or lane segment.

[0146] Additionally, whenever a human overrides the automatic control or the vehicle under automatic control gets stuck (i.e., does not progress beyond a first threshold, such as 1 minute or other time threshold), a -1 may be recorded as the first value, and otherwise (e.g., when the automatic control is not overridden and the vehicle does not get stuck), a 0 may be recorded as the second value. The average of the recorded first and second values ​​may be used as a measure of competence. The inverse may be used to learn the competence of a human driver. For example, when under manual control, the vehicle does not move beyond a second time threshold (e.g., 1 minute or other time threshold), a -1 may be recorded as the third value. For example, when the driver does not follow a prohibited next action identified by the lane-level route planner, a -1 may be recorded as the fourth value. For example, when the driver seamlessly proceeds into an adjacent lane segment, a 0 may be recorded as the fifth value. The average of the third, fourth, and fifth values ​​may be recorded as a measure of the human's competence to perform a next action when the vehicle is in the current lane segment.

[0147] As previously mentioned, semantic labels may be used to define the objective of the route calculated by the lane-level route planner. Examples of semantic labels include those described with respect to FIG. 5, such as “comfort” (and / or related semantic concepts), “urban” (and / or related semantic concepts such as “highway”), “highway” (and / or related semantic concepts such as “manual driving” and / or related semantic concepts such as “automated driving”). Fewer, more, other semantic labels, or combinations thereof are possible. To illustrate, if a user indicates a preference for comfortable roads, the lane-level route planner calculates the policy taking into account lane segments labeled as “comfort.”

[0148] Note that the mapping from purposes to semantic labels does not have to be a one-to-one mapping, so a semantic label may combine several purposes, or one purpose may be used to define one or more semantic labels.

[0149] In one aspect of the disclosed implementation, multi-objective deep reinforcement learning (MODRL) may be used. In reinforcement learning or deep reinforcement learning, a deep neural network (DNN) may be used to learn the policy. Alternatively or additionally, a DNN may be used to learn the value function. Therefore, reinforcement learning techniques may be used instead of planning techniques. In reinforcement learning, the objective may be learned via the DNN from feedback. The feedback may include one or more of the time to traverse a lane segment, human overrides to capabilities, or other feedback. In reinforcement learning, instead of learning probabilities, probabilities and costs are combined into an expected value for reaching the goal (i.e., destination).

[0150] In one aspect, TMDP may derive policies using a navigation map, as described herein. In another aspect, TPOMDP may derive policies using TPOMDP, which may enable beliefs about, for example, but not limited to, the level of traffic on a road, lane, or lane segment; the amount of capacity of a new road, lane, or lane segment; or other lane-level information as described herein.

[0151] FIG. 6 shows an overview 600 of learning and using a navigation map according to an implementation of the present disclosure. The lane level route planner may include a learning module that obtains lane level information. The lane level route planner may run in a vehicle, and the lane level information may be incorporated into a navigation map of the vehicle. The lane level information may be sent to a server, and the server may receive the lane level information from multiple vehicles. The server may incorporate the received lane level information into the navigation map. The navigation map may then obtain a policy (e.g., a route) using the lane level route planner.

[0152] Overview 600 shows three different routes 602, 604, and 606 taken (by the same or multiple vehicles, not shown) from an origin 608 to a destination 610. During travel, lane level information is collected for lane segments, such as lane segment 612. History 614 shows learned lane transitions, and the shading of the lane segments in history 614 indicates learned characteristics of the lane segments. History 614 includes other lane segment information learned as described herein.

[0153] The history of at least some (e.g., all) of the lane segments along at least some (e.g., all) of the route may be combined (e.g., counted) into a navigation map, as described herein. As described herein, multiple intents and other semantic labels may be recorded for each lane segment history, as indicated by box 618.

[0154] Based on the learning, a state transition graph structure (i.e., state transition function T described above) may be generated, as illustrated by transition graph 620. The state transition graph may be used by an SSP MDP or POMDP model, as described herein. A lane-level multipurpose lane-level route planner navigation map is then output (e.g., generated). The lane-level information of the navigation map includes averages of vehicle occupancy of lane segments. For example, the lane-level information may include averages of one or more of speeds, stops, perceived traffic density, battery usage, lane change success rate, etc.

[0155] Based on the learning, a cost (reward) function to be used in the SSP MDP or POMDP model can also be generated. The average of different objectives can be recorded as rewards by observing time, control transfer points, battery usage, speed, etc. as described above. From the reward function, it can be known whether slack is possible between the objectives. For example, if the expected value of the route is 10 minutes longer than the optimal, then a slack of at least 10 is possible. Such reward functions and slack can be used to set a preference ranking, as described with respect to FIG. 5 and shown in preference ranking 622.

[0156] 7 is a flow chart diagram illustrating an example of a technique 700 for learning a navigation map for route planning according to an embodiment of the present disclosure. The navigation map is used by a lane-level route planner. The technique 300 may be implemented partially or fully in an apparatus that may be a host vehicle, such as the vehicle 100 shown in FIG. 1, one of the vehicles 210 / 211 shown in FIG. 2, a semi-autonomous vehicle, another vehicle that may include driver assistance features, or a vehicle that is manually controlled by a driver, etc. The apparatus may be a server, which may be the communication device 240 of FIG. 2.

[0157] Technique 700 may be implemented as instructions (e.g., instructions) stored in a memory. The memory may be memory 134 of FIG. 1. The memory may be memory of a server. The instructions may be executed by a processor, such as processor 133 of FIG. 1. Technique 700 may be implemented using dedicated hardware or firmware. Multiple processors, memories, or both may be used.

[0158] In step 702, the technique 700 obtains lane level information for a road. The lane level information includes information for different segments of different lanes of a road. For example, a road may include a first lane and a second lane. The lane level information includes first lane information associated with the first lane and second lane information associated with the second lane.

[0159] The lane-level information may be obtained at a host vehicle, and the lane-level information may be used to evolve a navigation map of the host vehicle (e.g., used in, used by, available to the host vehicle). The lane-level information may be obtained (e.g., received) at a server that obtains lane-level information from multiple vehicles. The server may send the evolved navigation map to the vehicles for use by their respective lane-level route planners. The server may use the evolved navigation map to calculate routes (e.g., policies).

[0160] As discussed above, the lane level information may include at least one of environmental information, vehicle information, or human information. The environmental information may include at least one of traffic patterns, pedestrian patterns, or impassability information for the lane segment. The vehicle information may include a user's preferred route and a user's control transfer request.

[0161] In one example, obtaining lane level information of a road may include recording a first time of automated traversal of the road with a stop on the road, recording a second time of automated traversal of the road without a stop on the road, recording a third time of manual traversal of the road with a stop on the road, and recording a fourth time of manual traversal of the road without a stop on the road. In one example, obtaining lane level information of a road may include recording a following lane segment during manual traversal. In one example, obtaining lane level information of a road may include recording a human override of the automated cruise control. Automatic traversal of a road means that a vehicle is automatically controlled to traverse the road. Manual traversal of a road means that a vehicle is manually controlled (e.g., by a human) to traverse the road.

[0162] In step 704, the technique 700 converts the lane level information into a state transition function that can be used to obtain a route to the destination. As described above, the state transition function may be of a TMDP or TPOMDP model. The route to the destination may be, or may be encapsulated in, a policy, as described above. In one example, the policy, i.e., the route, is probabilistic.

[0163] In step 706, the technique 700 receives a destination. For example, a server (or, more specifically, a lane level route planner running on the server) may receive a current location and a destination (e.g., from a user device, a mapping application in the vehicle, etc.) to obtain a route. For example, a lane level route planner running on the device may receive a destination from a user (e.g., a driver or passenger), and the lane level route planner may obtain a route. For example, the lane level route planner may infer a starting location for the route based on the current location of the device, etc.

[0164] In step 708, the technique 700 obtains the policy as a solution of the model using the state transition function.

[0165] 8 is a flow chart diagram illustrating an example of a technique 800 for providing explanations for route planning according to implementations of the present disclosure. The technique 800 may be implemented in part or in full by a route planner, such as a lane-level route planner described herein. The technique 800 may be implemented by a module of the route planner, such as an explanation module. The lane-level route planner may provide actions that may be used to control a vehicle to automatically follow a route. The technique 800 may be used to provide explanations as to why a certain road, lane, or lane segment is selected by the route planner.

[0166] Technique 800 may be implemented as instructions (e.g., instructions) stored in a memory. The memory may be memory 134 of FIG. 1. The memory may be memory of a server. The instructions may be executed by a processor, such as processor 133 of FIG. 1. Technique 700 may be implemented using dedicated hardware or firmware. Multiple processors, memories, or both may be used.

[0167] The technique 800 is further described with reference to Figure 9. Figure 9 is a diagram illustrating an example 900 of providing illustration based on multiple objectives according to implementations of the present disclosure.

[0168] In step 802, the technique 800 may determine (e.g., calculate, obtain, etc.) a route based on at least two objectives received from a user. A second of the at least two objectives may be constrained within a margin value of a first of the at least two objectives. As discussed above, the margin value indicates how much deviation from the first objective is allowed to improve the second objective.

[0169] In one example, determining a route means obtaining a policy for navigating a route from a starting (e.g., current) location to a destination. The route may be determined using a Markov decision process, as described herein. A lane level planner may determine the route, as described herein. A lane level route planner may determine the route using a navigation map, as described herein. The route planner may be a multi-purpose route planner.

[0170] The at least two objectives may be any number of objectives. The at least two objectives may be related (constrained) in any manner as described herein. The at least two objectives may be related using a directed acyclic graph. In one example, the at least two objectives may include at least two of a time objective, a comfort objective, an autonomy objective, or a city objective.

[0171] 9 shows two objectives: a time objective 912 and an autonomy objective 914. The autonomy objective 914 is constrained (e.g., to be at) a margin value 916 of 3 minutes of the time objective 912. In one example, the at least two objectives may be default settings for the route planner. In one example, the technique 800 may receive the first objective, the second objective, and the margin value from a user.

[0172] In one example, the first objective, the second objective, and the margin value can be received from the user in a verbal command. For example, the verbal command can be "Find the fastest route that allows 3 minutes margin to increase autonomy." The verbal command can be converted via the route planner's semantic processing module into an internal representation that can be or resemble a directed graph 910. The semantic processor can identify the objectives, the relationships between the objectives, and any margin values ​​in the verbal command. For example, the semantic processor can convert the words "fastest route," "increase autonomy," and "allow 3 minutes margin" into a time objective 912, an autonomy objective 914, and a margin value 916, respectively.

[0173] In one example, a user may be provided with a graphical user interface that allows the user to specify objects and margin values. Examples of graphical user interfaces are described with respect to FIGS. 10 and 11. The graphical user interface may be provided on a display, such as a vehicle display or a display of a user's device. The graphical user interface may provide a drag-and-drop interface for the user to drag icons (e.g., bubbles, boxes, words, etc.) that represent available constraints and connect the icons to form a DAG as described with respect to FIG. 5. By clicking on the connection between two constraints, the user may provide margin values. In one example, a semantic processor of the route planner may convert the DAG into a description that may be displayed on a display. In one example, the user may be limited to forming a DAG that is a list (e.g., a set of objectives).

[0174] In one example, the user's verbal commands may be displayed textually on the display. In one example, a DAG representing the user's verbal commands may be displayed on the display.

[0175] In step 804, the technique 800 receives a request from a user for clarification regarding actions along the route. The request may be for clarification regarding actions that have not yet been performed or actions that have already been performed.

[0176] The determined route may be displayed on a display. The route may be overlaid on a map, such as shown in map 920 of Figure 9. Map 920 includes a route 922 from an origin 923 to a destination 924. Origin 923 may be the current location of the vehicle and may be updated as the vehicle traverses route 922.

[0177] The map 920 indicates that the next segment to be taken is lane segment 929. The map may also include controls that the user can use (e.g., press) to obtain information or obtain an explanation. For example, the map 920 includes an alternative lane segment selector 926 that the route planner could select. In response to the user pressing the explanation control 925, the technology 800 provides the explanation. The information control 927 may be used by the user to request information about the lane segment, lane, or road next to which the information control 927 is displayed. In response to the information control 927 being pressed, the technology 800 may provide information about the lane segments available on the navigation map, as described above. Other arrangements or visual depictions of the explanation control 925, the alternative lane segment selector 926, and the information control 927 are also possible.

[0178] In one example, the technique 800 may receive a request for clarification via a verbal request from a user.

[0179] A request for clarification may be a request regarding a particular possible action in a current state. For example, a user may ask, "Why not go right?" to receive an explanation regarding the next immediate action selected by the route planner. In such a request, the route planner provides an answer by comparing the selected action with the particular possible actions in the route planner's current state.

[0180] A request for an explanation may be a request that includes all possible actions in the current state. For example, a user may ask, "Why are we going straight?" to receive an explanation for the next immediate action selected by the route planner. In such a request, the route planner provides an answer by comparing the action selected in the route planner's current state with other possible actions in the current state. In an example, a user may select (e.g., click) a particular point on the map, such as point 928, from which the route planner may select from various possible actions.

[0181] The user may request an explanation for an action already performed or selected to be performed by the route planner. For example, the user may ask, "Why didn't you go right?" In another example, the user may ask, "Why did you go straight?"

[0182] In one example, the user may manually override the action. The user can manually override the action by selecting (e.g., pressing) the alternate lane segment selector 926 or by issuing a verbal command such as "go right next." The user can manually override the action by manually controlling the vehicle. The user can override the action before or after receiving the instructions.

[0183] When a user selects a point on the map 920 and requests an explanation, the location on the map is converted to a state, s, which may be as described above for any route planning model.

[0184] In step 806, the technique 800 provides an explanation to the user. The explanation may describe (e.g., include) the extent to which the margin value is violated.

[0185] When computing a route (e.g., a policy), and for at least some objectives, the route planner calculates, as described above, for at least some states (e.g., lane segments) and possible actions, the respective expected discounted rewards Q o We may maintain (e.g., store) Q(s,a) where o is an objective, s is a state, and a is an action. Q(s,a) = [Q1(s,a), ..., Q k (s, a)] is a vector of 1,..,k objective expected discounted rewards for a particular state s and action a. Obtaining the expected discounted reward depends on the policy. Q o (s,a) is the expected discounted reward for objective o if one takes action a once in state s and then follows the policy. Therefore, each cost of taking a lane segment transition from the current lane segment can be maintained for at least one of at least two objectives.

[0186] In one example, the route planner may store gain values ​​according to how much change occurs between states and for each objective. The top states (e.g., lanes or lane segments along the route) with the largest difference in value may be selected as the explanation.

[0187] To illustrate, the expected discounted reward values ​​Q may be sorted in state s relative to the expected discounted reward values' actions. In one example, the values ​​may be stored separately for each objective. For objective i, the ordering relative to the actions is Q i (s,a2)>Q i (s,a4)>Q i (s,a3)>Q i(s, a1), where a1, a2, a3, and a4 are possible actions at state s. Therefore, action a2 can be considered as the optimal action for objective i. However, for another objective j, the ordering is Q j (s,a4)>Q j (s,a1)>Q j (s,a2)>Q j (s, a3). Therefore, action a4 can be considered as the optimal action for objective j. Furthermore, assume that action a4 is the final selected action for state s. Therefore, the expected discounted reward for objective i is Q i (s,a2)-Q i It decreases by (s,a4).

[0188] An explanation can be provided using the semantic meaning of the actions a1, a2, a3, and a4 and the states. The state s can have a meaning as a map location. For example, the state s can be a GPS point in the map. For example, the state s can be a GPS point at an intersection in the map. The action a can also have a meaning like turn left, turn right, go straight, make a U-turn at the intersection, etc. So, an explanation sentence with placeholders (or variables) can be constructed. For example, a sentence could be "<action a> was selected in <state> because the expected time i (s,a optimal )-Q i An example of such a sentence might be: A U-turn was chosen at the junction of Barrows and Watson because it would have increased the expected time by only 1.2 minutes, and it is assigned a margin of 3.0.

[0189] ​To illustrate, a user may ask, "Why didn't you go right?" That is, the user is asking why route 942 on map 940 was not selected by the route planner. Technique 800 determines that state s is a GPS location at point 928. Technique 800 determines (e.g., calculates, retrieves, obtains, looks up from memory or storage) an expected discounted reward value for each objective and each action that the user seeks to compare (e.g., describe). In this manner, technique 800 determines payoff values ​​for each objective of time and autonomy and for each action of go_straight and turn_right. Thus, technique 800 determines Q corresponding to route 922. time (s,a=go_straight) and Q autonomy (s,a=go_straight) is determined, and the Q corresponding to route 942 is time (s,a=go_right) and Q autonomy Determine (s,a=go_right), where s=the GPS position at point 928.

[0190] For illustration purposes, assume the following values ​​have been determined:

number

[0191] That is, route 922 may be determined to take 2 minutes to travel, and the vehicle may drive autonomously along route 922 for 1 minute; route 942 may be determined to take 6 minutes, and the vehicle may drive autonomously along route 942 for 4 minutes.

[0192] Even though route 942 would result in a more autonomous trip that satisfies autonomy objective 914, technique 800 determines that route 942 is not within margin 916. The time difference between routes 942 and 922 is Q time (s,a=go right )-Q time (s,a=go straight ) = 6 - 2 = 4 minutes. This time difference (4 minutes) is not within the margin (3 minutes).

[0193] Expected discounted rewards are calculated for conditions along the alternative route. Thus, the user may request an explanation for the conditions along the route or the alternative route. When planning a route, a large number of contingency plans are considered probabilistically. Thus, an explanation may be provided as to why another given route is not selected. For example, in response to a user selecting an alternative route, the explanation module may use the above sorting algorithm, excluding only the lanes / roads of the difference between the selected route and the alternative route. Thus, it may identify the cause of the largest cost difference between objectives between the routes.

[0194] Next, the technique 800 can construct an explanation. In one example, an explanation module of a route planner can construct the explanation. The explanation may include a descriptor for each of the first action, the second action, the first objective, the second objective, and the degree to which the second action violates the margin value.

[0195] In one example, constructing an explanation may mean replacing (e.g., inserting) a placeholder in a template with a descriptor. In one example, a template may have the format "The reason for taking <action> is because <violated objective> is outside the allocated margin of <margin> to improve <violated objective> by <alternative action>," where <action taken>, <alternative action>, <violated objective>, <margin>, and <violation amount> are each placeholders for a respective semantic descriptor.

[0196] The placeholder for <action taken> may be a placeholder for the action selected by the route planner, the placeholder for <alternative actions> may be a placeholder for one or more alternative actions, the placeholder for <violated objective> may be an objective whose value due to one of the alternative actions is not within a margin of the optimal value, <margin> may be a margin value, and the <violating objective> may be an objective whose value causes the value of the violated objective not to be within a margin. Thus, using the example above, the explanation might be "The reason for going straight is because a right turn would cause time outside the allocated margin of 3 minutes to improve autonomy."

[0197] In one example, multiple templates may be available and the explanation module may select one of the templates. In one example, a template may not include one or more of the placeholders above. A template may be "The reason we went to <action taken> is because <conditions on lane segment>" or "The reason we did not take <action not taken> is because <conditions on lane segment>". To illustrate, if a user provides a primary objective of maximizing autonomy and the route planner selects route 942 on map 940, the user may ask "Why not go straight?" The route planner selects route 942 because the navigation map includes information indicating that lane segment 944 typically contains traffic congestion and therefore the vehicle is not capable of traversing the lane segment autonomously. The explanation may be "The reason we did not go straight is because we are not capable of traversing the lane autonomously". In one example, the lane segments may be simultaneously highlighted on map 940 to indicate to the user which lane segment the explanation pertains to. As discussed above, in one example, capability information in the navigation map can be learned. In one example, the capability information may be pre-loaded (eg, pre-configured) within the navigation map.

[0198] In one example, the instructions may be provided (e.g., output to a user) in one or more modalities. The instructions may be provided in at least one of a visual, tactile, or auditory modality. In a visual modality, the instructions may be displayed to the user, such as on a vehicle display. In an auditory modality, the instructions may be read aloud to the user.

[0199] Haptic feedback may be useful, for example, to hearing and / or visually impaired users. Haptic instructions may be provided by vibrating one or more physical components of the vehicle (e.g., the steering wheel, the seats, the pedals, etc.).

[0200] Different vibration patterns and locations (e.g., left side of the steering wheel, right side of the steering wheel, left side of the seat cushion, right side of the seat cushion, both sides of the steering wheel, both sides of the seat, etc.) can be used to convey different aspects of the instructions. In one example, a steering wheel may be used in situations where a sensor in the steering wheel detects that a person is holding the steering wheel.

[0201] Different vibration patterns can be used to convey different values ​​of the template placeholders. The haptic description may be in the format of "<action taken vibration pattern>---<alternative action vibration pattern>---<violated intent vibration pattern>---<violating intent vibration pattern>". Vibration patterns are represented here using "R" and "-", where "R" may represent a vibration or vibration tone and "-" may represent a pause, or alternatively, "R" may represent a high intensity vibration or vibration tone and "-" may represent a low intensity vibration or vibration tone.

[0202] A distinct vibration may be available for each action to be performed. For example, the actions "move right", "move left", "go straight", and "move diagonally" may be represented by the vibration patterns "RR", "R-", "-R", and "--", respectively. Similar patterns may be available for other placeholder values. An explanation may be structured as a sequence of vibrations and pauses, with each subset of vibrations and pauses representing a placeholder value. The vibration and pause sequence may be preceded by a sequence that signals (e.g., indicates) the start of an explanation.

[0203] FIG. 10 illustrates an example of a user interface 1000 for a multi-objective setting for a lane-level route planner according to an implementation of the present disclosure. The user interface 1000 includes a vehicle user interface, such as the user interface 135. The user interface 1000 includes a first object 1010 representing a first objective and a second object 1020 representing a second objective. Here, the representation includes a third object 1030 representing the third objective since the third objective is included in the multi-objective policy. The placement of the first object 1010 and the second object 1020 in the representation on the user interface 1000 indicates the priority (e.g., relationship, etc.) of the first objective relative to the second objective. Similarly, due to the inclusion of the third objective, the placement of the third object 1030 in the representation on the user interface 1000 further indicates the priority of the third objective relative to at least one of the first objective or the second objective.

[0204] More specifically, in the arrangement of FIG. 10, a first object 1010, a second object 1020, and a third object 1030 are arranged in a list with the first object 1010 at the bottom of the list, the second object 1020 at the top of the first object 1010 in the list, and the third object 1030 at the top of the second object in the list. This arrangement shows that the first objective is constrained by the second objective, which is constrained by the third objective. Each object 1010, 1020, 1030 may include a respective objective descriptor. In this example, the first objective is related to speed or time to complete the goal and has a "time" descriptor, the second objective is related to the user's preference for a comfortable lane and has a "comfort" descriptor, and the third objective is related to the user's preference for maximizing autonomous driving and has an "autonomy" descriptor. In another example, as described above with respect to FIG. 9, the order of objectives may be such that unconstrained objectives are at the top of the list.

[0205] The priorities may also include margin values. Via controls 1024 and 1034, the user can set margin values ​​1024 and 1032, respectively. In this example, controls 1022 and 1034 are shown as sliders; however, other types of controls are possible.

[0206] In this example, the first object 1010, the second object 1020, and the third object 1030 may each be a drag-and-drop object. Feedback in the form of moving any of the objects in the graphical user interface constitutes a change in the multi-objective policy. For example, swapping the positions of the second object 1020 and the third object 1030 will swap the second object and the third object. Also, one or more of the objects 1010, 1020, 1030 may include a pull-down menu to select one objective from a group of candidate objectives to associate with the respective object. The pull-down menu may allow a change in the lane-level route planner multi-objective policy to include changing the first objective to another candidate objective of the group, changing the second objective to another candidate objective of the group, changing the third objective, if present, to another candidate objective of the group, adding a new objective from the group with a priority of the new objective relative to another objective, removing an objective, or some combination of these changes.

[0207] FIG. 11 illustrates another example of a user interface 1100 for setting multiple objectives of a lane-level route planner in accordance with an implementation of the present disclosure. The multiple objectives of the user interface 1100 are arranged in a directed acyclic graph. The user interface 1100 includes a vehicle user interface, such as the user interface 135. The representation includes a first object 1120 representing a first objective (i.e., "time"), a second object 1130 representing a second objective ("comfort"), and a third objective 1110 representing a third objective ("autonomy"). The arrangement of the first object 1110, the second object 1120, and the third object 1130 in the representation on the user interface 1100 indicates the relationship (e.g., priority) of the objectives.

[0208] In the user interface 1100, the second object 1120 and the third object 1130 are positioned adjacent to each other in a line parallel to the top of the graphical user interface, with the first object 1110 located below the second object 1120 and the third object 1130. This graphically represents the priority of the first objective over the second objective and the priority of the third objective over the first objective. The first object 1110, the second object 1120, and the third object 1130 may each be a drag-and-drop object as shown, but may also be other objects that can be manipulated via feedback to change the multi-objective policy. For example, one or more of the objects 1110, 1120, 1130 may include a pull-down menu. The objects 1110, 1120, 1130 may represent other objectives in different implementations.

[0209] The representation of FIG. 11 may use objects representing objectives formed similarly to those of FIG. 10, i.e., objects including embedded objects that may be used to adjust the relative priority between two objectives of a multi-objective priority (e.g., embedded object 1024 of second object 1020 or embedded control 1034 of third object 1030). However, FIG. 11 shows a different arrangement that does not use objects 1110, 1120, 1130 of each objective to adjust the relative priority. Instead, this representation includes a fourth object 1140 that identifies a margin value and modifies a first margin value in response to a touch. The fourth object 1140 is positioned between the first object 1110 and the second object 1120 in the user interface 1100. This representation also includes a fifth object 1150 that identifies a second margin value and modifies the second margin value in response to a touch. The fifth object 1150 is positioned between the first object 1110 and the third object 1130 in the user interface 1100 .

[0210] The fourth object 1140 and the fifth object 1150 in this example may each be a pull-down menu, i.e., upon contact, a menu may be displayed containing possible new first or second margin values, as appropriate. One or more of the objects 1140, 1150 may be a slider or other object that is manipulable on the user interface 1100 to change a value associated therewith and provide that value to a processor for inclusion in an updated multi-purpose policy.

[0211] Other indicators of relationships between objectives may also be used in the representation of a multi-objective policy. For example, the representation in Figure 11 includes an arrow pointing from the second objective 1120 to the first objective 1110 and an arrow pointing from the third objective 1130 to the first objective 1110, which indicates a constraint relationship between the respective related objectives.

[0212] 12 is a diagram illustrating an example map illustrating the use of a standard definition (SD) map for route planning in autonomous driving according to implementations of the present disclosure. In map 1200, a route 1202 is planned by a lane level route planner using an HD map. A planner, such as a lane level route planner as described herein, may have selected route 1202 because, for example, navigation map information determines that the AV is capable of autonomously driving the lane segments of route 1202. Capability on a lane segment may be indicated by a probability, a threshold, a tolerance, etc.

[0213] Portion 1204 shows unmapped side roads that the planner did not consider because the planner may consider that the AV is not capable of autonomously driving on unmapped roads. Additionally, such side roads may not be available (e.g., not included) in the HD map. However, if the AV is capable of driving on portion 1204, using portion 1204 may lead to a shorter route (e.g., one minute shorter). Therefore, it may be beneficial to explore and map unmapped portion 1204.

[0214] Thus, in an aspect of the present disclosure, the planner may be configured to explore unmapped roads to add HD information for unmapped roads. "Exploring" as used in this context may mean that the route planner can use roads in the SD map that are not in the HD map for route planning and the vehicle may be controlled to traverse these unmapped routes. In another example, the AV may be guided to traverse roads in the SD map. For example, a user (e.g., a teleoperator, etc.) may send instructions to the vehicle to traverse roads in the SD map. While the AV is traversing unmapped roads, HD information may be collected using sensor data. The HD information may be used to build or extend an HD map for the AV. New road and lane segment information may be added to the navigation map by exploration. At least a partial HD map may be built by exploration. Information regarding lanes, curbs, speed bumps, stop lines, traffic signal positions (in three-dimensional space), etc. may be added to or constitute the HD map. In some cases, this information may be the same as that which may be provided by a commercially available (e.g., purchased) HD map. The quality of the acquired information may depend on the vehicle's sensors and detection and identification algorithms. However, even noisy information of these kinds of HD features may be useful for decision-making and trajectory planning for future passages of explored roads in SD maps. One passage of a road in SD maps may be sufficient for later lane-level route planning for autonomous driving and passage. More passages may improve the quality of the HD maps used by decision-making and trajectory planning.

[0215] 13 is a diagram of a system 1300 for searching for new roads to improve a map according to implementations of the present disclosure. The system 1300 may include a planner 1302, a mapper 1304, and a safety decision component (DC) 1306. Other implementations of the system 1300 are also available. In some implementations, additional components (e.g., elements, modules, etc.) of the system 1300 may be added, certain elements may be combined, and / or certain elements may be removed.

[0216] System 1300 or components therein may be implemented in a vehicle, such as vehicle 100 of FIG. 1. The vehicle may be an autonomous or semi-autonomous vehicle. System 1300 or components therein may be implemented as executable instructions that may be stored in a memory, such as memory 134 of FIG. 1, and executed by a processor, such as processor 133 of FIG. 1. One or more of the components of system 1300 may be implemented by dedicated hardware arranged to perform the respective information processes described below.

[0217] Planner 1302 may be, or may be part of, a lane-level route planner. Planner 1302 may weigh the cost of searching SD map roads (e.g., compared to the cost) of planning using only HD map roads. Planner 1302 may be an SSP MDP route planner, and this weighing may be incorporated or included in the route planner's cost function. Planner 1302 may be a POMDP route planner, and the cost may be an explicit belief. The state space of the POMDP model is S = S r ×S t where S r is a set of roads, and S t indicates how passable the road is. Action space A is a set of actions for selecting a subsequent road. The subsequent road can be a road in the HD map (e.g., a lane segment) or a road in the SD map.

[0218] The SD map is a state space Sr Initially, before any road in the SD map has been traversed at least once, none of the lanes of such road are yet known. t ), SD roads may be initially uncertain. For example, the traficability of any given SD road may be set to a small value. The planner can balance the tradeoff between exploration and exploitation through POMDP planning so that the vehicle is automatically controlled on roads in the SD map.

[0219] In an online lane-level route planner, which may update routes at a predetermined frequency (e.g., 1 Hz or other frequency), the road traversability S t may be updated continuously. The traversability may be updated with the same frequency as the route. The lane-level route planner may consider the current roads (i.e., conditions) in the map based on GPS information every predefined number of seconds (e.g., 1 second) and perform planning for a predefined amount of planning time (which may be exactly a predefined number of seconds). After the predefined amount of planning time, the lane-level route planner may provide the optimal route obtained by the lane-level route planner (based on the calculated policy). If the vehicle is controlled to move incrementally, the lane-level route planner may obtain new roads (which may be the same as the previous roads) and update the previous route plan.

[0220] For offline lane-level route planners, the passability S t may be updated. The offline lane-level route planner may be a cloud-based service that provides route planning.

[0221] In one example, the planner may be configured to explore unmapped roads based on an objective provided by a user. The planner may obtain an objective indicating that a standard definition map is to be used in addition to a high definition map to obtain a route. This objective may be implicit or explicit. The route planner may be configured to choose to explore new roads and weigh the trade-off between reaching a customer's goal and trying roads that the route planner determines the AV may be controlled to travel on.

[0222] For example, a user may indicate a preference for routes that are within 2 minutes of the fastest route. The fastest route is calculated based on the HD map first. The planner may use a 2 minute margin value as an implicit objective to consider unmapped roads. In one example, a user may provide an explicit objective related to unmapped roads. For example, an available objective that a user can set may be a "search objective." A user may provide a search objective as described herein. For example, a user may provide (e.g., build) a directed graph of objectives similar to directed graph 910 of FIG. 9 where autonomy objective 914 may be replaced with a search objective. In one example, a user may provide an objective using a user interface as described with respect to FIGS. 10 and 11. If the margin value is set to 0, the planner obtains a route using only HD routes.

[0223] The mapper 1304 records data from HD sensors of the vehicle, such as sensor 136 of FIG. 1. The HD sensor data may be recorded as the vehicle travels along roads of the SD roads before the roads are mapped and added to the navigation map. For example, the HD sensor data may include point clouds, images, GPS coordinates, tracked objects, and the like. In one example, data from sensors of other vehicles traveling along roads of the SD map may be obtained, such as at a central server. Data from sensors of other vehicles may also be used to obtain HD map data. Thus, more generally, regardless of the source of the sensor data, HD map information may be obtained for roads of the SD map based on the sensor data. The HD map information obtained at the server may be transmitted to the vehicle.

[0224] Thus, mapper 1304 can use HD sensor data collected while traversing an SD lane or SD route segment. The HD sensor data can be said to become an HD map for that SD lane or SD route segment. In one example, an SD lane or SD route segment can be traversed (e.g., driven) by human control of one or more vehicles equipped with HD sensors to obtain an HD map for the SD lane or SD route segment. The obtained HD map can be improved by revisiting (i.e., retraversing) the SD lane or SD route segment.

[0225] The sensor data may be used to identify road characteristics. For example, detection, appearance, or disappearance of an object may be detected from the sensor data. For example, the data may indicate, for each detection segment of the road (e.g., 1 meter, 2 meter, or some other detection segment) of the SD road, the presence or absence of an object, the identity (e.g., type) of the object (if recognized using image processing or data analysis, etc.), the presence or absence of an obstacle, the location of the object or obstacle (e.g., left, right, center, or some other location on the road), the duration of the object's presence before it disappeared, etc.

[0226] Data collected from multiple traversals (e.g., vehicle traversals or multiple traversals) of roads in an SD map can be aggregated to obtain state probabilities that can be used to obtain routes for autonomous driving on SD roads. Example probabilities may include that an object appears at a first location with a probability of 0.823, that an object appears at a second location with a probability of 0.1, that an object at a third location moves at an average speed of 3.5 miles per hour, that an object at the third location is classified as a child with a probability of 0.9, etc. Such probabilities can be used for trajectory planning. Probabilities can be compounded over the length of a road or road segment to calculate a trajectory for traversing the road.

[0227] The probabilities may be initialized to values ​​that indicate uncertainty (i.e., lack of knowledge). The probabilities may be updated based on sensor data collected during traversal of roads in the SD map.

[0228] Information regarding types of decision-making components (DCs) that may be encountered on roads of the SD map may be identified and recorded using sensor data. Information regarding the decision-making component (DC) types may be required (e.g., instantiated) to traverse the road. Examples of DC components may include DC components for crosswalks, intersections, parking lots, etc. Decision components are described in U.S. Patent Application No. 16 / 696,235, filed November 26, 2019, and U.S. Patent Application No. 16 / 778,890, filed January 31, 2020, the entire disclosures of which are incorporated herein by reference.

[0229] The decision component may be directed (or may be able to resolve) a particular aspect of a scenario (e.g., a motion scenario) in autonomous driving. The DC may provide actions to control the AV.

[0230] The decision component may be an instance of a decision problem. The decision problem describes a potential pairwise interaction problem in a scene. The decision problem may be a specific abstract problem of autonomous driving that is known in advance and solved offline to provide an action of autonomous driving (e.g., stop, nudge, start, overtake on the right, overtake on the left, etc.) considering the current state of the environment. The decision component may be created online during driving (i.e., instantiated from the solved decision problem) when a motion scenario that the decision problem solves is encountered. In one example, the decision problem and the corresponding decision component may be as described below with respect to a scenario-specific motion control evaluation module and its instance. Examples of motion scenarios include a pedestrian scenario, an intersection scenario, a lane change scenario, or other vehicle motion scenario or combination of vehicle motion scenarios corresponding to an external object. In one example, the decision component may be a Markov decision process or a partially observed Markov decision process.

[0231] The scenario-specific motion control evaluation modules may be models, such as partially observable Markov decision process (POMDP) ​​models, of the respective motion scenarios, i.e., each model may be configured to handle a particular scenario. The motion management controller of the autonomous vehicle may create each instance of the scenario-specific motion control evaluation module in response to detecting a corresponding motion scenario.

[0232] In other words, the POMDP for a particular operating scenario may be solved in advance (e.g., offline) to generate a policy. The policy may include one or more actions for controlling the AV given currently observed and / or predicted conditions. When a particular operating scenario is encountered during autonomous driving (e.g., online), a POMDP may be instantiated, and the policy may be used by the instantiation to provide, at each time step, a respective action for controlling the AV based on the observed and / or predicted conditions at that time step.

[0233] The autonomous vehicle motion management controller may receive candidate vehicle control actions from each instantiated scenario-specific motion management evaluation module (SSOCEM) instance, identify a vehicle control action from the candidate vehicle control actions, and control the autonomous vehicle to navigate the portion of the vehicular traffic network according to the identified vehicle control action.

[0234] The DC information determined based on the HD sensor and the newly generated HP map of the road in the SD map can be used for subsequent planning by the lane-level route planner, as described above, to predict travel times based on the DCs the vehicle may encounter on the road. Thus, the vehicle may include a DC, for example, within the vehicle's memory, as executable instructions stored in the memory, or as dedicated hardware (not shown).

[0235] Safety DC 1306 can make risk-aware decisions. Safety DC 1306 can obtain risk-aware decisions by monitoring careful navigation on SD map roads towards HD map roads. The overall purpose of Safety DC 1306 is to provide actions that constrain the vehicle's trajectory on SD map roads to ensure safe passage by considering belief-based interactions when obstacles are present and may move to block the path and interact with the vehicle. Actions provided (e.g., selected) by Safety DC 1306 can be provided to a controller, such as controller 130 of FIG. 1, to control actuators of the vehicle to traverse the SD map roads. Because SD map roads may be dangerous to traverse autonomously due to lack of HD map details, Safety DC 1306 can be used to determine whether to continue or stop traversing the road autonomously. If Safety DC 1306 determines that the road should be stopped from being traversed autonomously, a teleoperator request is initiated, whereby the teleoperator may remotely control the vehicle to traverse the remainder of the road. In one example, the Safety DC 1306 may instruct a user of the vehicle (e.g., the driver) to take manual control of the vehicle.

[0236] When a vehicle is traveling on a road, a safety DC 1306 may determine at each time step whether it is safe to continue traveling on the road. The safety DC 1306 may be a Markov decision process, such as an MDP or a POMDP.

[0237] Safety DC 1306 may determine the speed for the next time step and may select an action (e.g., go left, go right, go straight, stop, etc.). Safety DC 1306 may select the speed or direction based on the direction of the nearest HD map lane.

[0238] In one example, the safety DC 1306 may determine an action to traverse a drivable area of ​​a road. The drivable area of ​​a road is an area of ​​the road where the AV can be safely operated. The determination of the drivable area may be as described in PCT Patent Application No. PCT / US2018 / 035441, filed May 31, 2018, the entire disclosure of which is incorporated herein. Thus, obtaining the drivable area may be as follows: First, the drivable area may include an area where the vehicle is not predicted to be safe to travel (e.g., the entire road within a look-ahead window). Based on the vehicle's sensor data, static and dynamic (e.g., moving) objects may be detected (e.g., sensed, observed, etc.). The area where the vehicle is not predicted to be safe to travel may be clipped (e.g., excluded, segmented) from the road. The portion of the road where the static object is located (e.g., observed, detected) is first removed from the drivable area. The drivable area may be further adjusted for dynamic objects. Based on the respective predicted trajectories of each dynamic object, a portion of the drivable area may be further clipped. The future position of the vehicle may be evaluated against the future predicted (e.g., expected) positions of the dynamic (i.e., identified moving) objects. The drivable area of ​​the vehicle may then be adjusted to remove areas in the drivable area that correspond to the (e.g., overlapping) positions of the dynamic objects.

[0239] The drivable area may be further adjusted by forming a tunnel around the vehicle, through which the vehicle is controlled to pass through the road. The tunnel may be considered as further restricting the drivable area. The left and right boundaries may extend laterally along the left and right sides of the vehicle. The left and right boundaries may be defined distances from the left and right sides of the vehicle. The defined distances may be based on a desired clearance of the vehicle from possible external objects. For example, the doors of a parked vehicle may open. The maximum size of a car door is about one meter, so the defined distance between the vehicle and each of the left and right boundaries may be one meter. The defined distances may be based on a predefined spacing tolerance. The defined distances may vary, for example, to take into account the presence of an external object. For example, the defined distance may be a first value when no external object is detected by the vehicle, and a second value smaller than the first value when an external object is detected. The left and right boundaries may be away from the vehicle, i.e., at different distances (e.g., if there is an external object on one side of the vehicle but not on the other, or if there are two different external objects on opposite sides of the vehicle, resulting in two different defined distances).

[0240] The Safety DC 1306 may be used to navigate the roads of the SD map (while learning the HD map from the sensors). The Safety DC 1306 may provide constraints between time steps (e.g., second to second) of the trajectory along the roads.

[0241] The state space of safety DC1306 is S = S l ×S m ×S b where S l is the set of locations along roads in the SD map, and S m indicates whether motion (e.g., motion of dynamic objects such as other vehicles, pedestrians, bicycles, etc.) is detected at those locations, and S bindicates whether obstacles (e.g., static objects) are detected at those positions. In other words, the state space S is a set of positions S m If there is motion in and its position S b If there is an obstacle in the SD Map S l Consider a position along a road of Note that the lanes of the road are not yet known (e.g., not mapped).

[0242] The action space of Safety DC1306 is A = A m ×A o which takes into account the action and direction. m A describes actions such as "stop," "move forward," and "forward." o describes a directional action such as "left of lane", "center of lane", "right of lane", etc.

[0243] Regarding the transition function of the model, the state coefficient S l can transition forward with a probability proportional to the distance or speed limit. For example, the maximum travel time for travelling and inching through the map is t max (e.g., 42.23) seconds, then we can use this maximum transit time as a normalization scale so that the probability of the maximum transit time transition is 1. All other transitions with transit time t have probability t / t max Similar calculations can be performed using other variables such as speed limits or road distances. Alternatively, the transition probabilities may be set to a constant 1.0 (during the planning phase) which indicates always proceed.

[0244] Motion detection status coefficient S m , and the obstacle detection state coefficient S bcan be derived from historical data on the volume of arrivals or other forms of road users. For example, if a bike lane is detected on the road being searched (or a bike lane is already recognized in the SD map), and previously traveled routes show that roads with bike lanes have a 0.123 chance of bike arrival per minute, then such historical information can be used to determine the transition S from no bikes to bikes present. m That is, (1-0.123=0.877) may be the remaining probability of no bicycle. b is the state coefficient S m , but is used for static versions of certain types of objects. The same principles can be applied to vehicles, pedestrians, and other objects that can be detected.

[0245] Condition factor S m and S b can have binary values ​​of 0 and 1 for moving and occluding objects, respectively. m and S b A binary value of may be sufficient to estimate safety and passability. In another example, additional or alternative state variables may be used. For example, a state variable indicating the expected number of objects of a certain type (e.g., concentration) may also be used. Such state variables may have integer values ​​(e.g., 0, 1, 2, ...) and may be modeled to follow a Poisson / exponential or birth-death process. For example, the state variable S ve , S pe and S bi may indicate the expected number of vehicles, bicycles, and pedestrians in the scene, respectively.

[0246] The reward function may be based on observations of the vehicle after an action is executed. For example, the reward may be based on whether an object is detected, whether an object is detected to the right, whether an object is detected to the left, whether the path is obstructed, whether the detected object is classified (e.g., recognized), whether the detected object is deemed dangerous, fewer criteria, more criteria, or a combination thereof.

[0247] In other words, the state indicates a position along the SD map whether there is a perceived feature (such as a moving object, road blockage, obstacle, other object, etc.). For example, there may be a building on the right side of the road and a tall fence on the left side of the road. These features may otherwise be available in the HD map, but may not be available in the SD map. Thus, the vehicle's sensor data may be used to recognize these features (such as obstacles). The Safety DC 1306 may determine an action (e.g., stop, inch, or go) at each point on the SD map (e.g., every 10, 20, 100 meters, or some other distance) along with the direction of the action (e.g., left, right, center).

[0248] In one example, the safety DC 1306 may use goal uncertain stochastic shortest path (GU-SSP) instead of POMDP. GU-SSP may be used to model route planning and decision making in stochastic environments with goal uncertainty. GU-SSP is described in S. Saisubramanian, KH Wray, L. Pineda and S. Zilberstein, “Planning in Stochastic Environments with Goal Uncertainty,” 2019 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 2019, pp. 1649-1654, doi:10.1309 / IROS40897.2019.8967718.

[0249] 14 is an illustration of an example 1400 of mapping roads in an SD map according to an implementation of the present disclosure. The example 1400 shows how an AV 1404 maps roads in an SD map using sensors in an AV 1402. Data from the AV's sensors can be used to detect HD information for roads in the SD map.

[0250] Example 1400 shows a map that includes a road 1404 that is not included in the vehicle's HD map. That is, road 1404 is not mapped. However, road 1404 is included in the vehicle's SD map. An initial width or lanes of road 1404 may be estimated from the SD map. Width and lanes may be estimated using any lane recognition technique. Circles, such as circle 1406, indicate locations along the road where a determination of safety DC 1306 in FIG. 13 provides action to a trajectory planner of AV 1402.

[0251] 13, an object 1408 is detected. A particular motion scenario associated with the object 1408 may be identified, and a corresponding DC is instantiated to handle (e.g., handle, avoid, bypass, etc.) the object 1408. For example, the object 1408 may be determined to be a parked vehicle indicating that the motion scenario is a "pass vehicle scenario," and a corresponding DC is instantiated to bypass the object 1408. Regions 1410A, 1410B may be recognized as non-drivable regions and are calculated from the HD sensor of the vehicle 1402.

[0252] Trajectory 1412 shows the path taken by vehicle 1402 on road 1412. Trajectory 1412 may be determined by the vehicle's decision-making process, which may be or may include at least planner 1302 and safety DC 1306 of Figure 13. The decision-making process selects an action for controlling vehicle 1402. The action may be one of those described above, such as go right, go left, go straight, stop, inch by, etc.

[0253] FIG. 15 is a flow chart illustrating an example of a technique 1500 for route planning in automated driving of an autonomous vehicle according to implementations of the present disclosure. The technique 1500 may be implemented by or within the autonomous vehicle. The autonomous vehicle may be a semi-autonomous vehicle. The technique 1500 may be implemented as instructions (e.g., operations) stored in a memory, such as memory 134 of FIG. 1. The instructions may be executed by a processor, such as processor 133 of FIG. 1. The technique 300 may be implemented using dedicated hardware or firmware. Multiple processors, memories, or both may be used.

[0254] In step 1502, the technique 1500 obtains an indication that a standard definition map should be used in addition to a high definition map to obtain a route. The indication may be obtained from a user (e.g., driver, passenger) of the AV. In one example, the indication may be obtained from a teleoperator. In one example, the indication may be provided as an objective to a route planner to obtain a route. The route planner may be a multi-purpose route planner as described herein. The objective may be provided using a user interface, for example, as described herein with respect to Figures 10 and 11. In one example, the indication may be provided via a verbal command that may essentially convey the command "search for unmapped roads."

[0255] In step 1504, the technique 1500 obtains a route for autonomously driving the vehicle to the destination. The route includes roads of a standard-definition map. The route may be obtained using a Markov decision process (MDP). The Markov decision process may be a POMDP. The Markov decision process may be a POMDP. The MDP may be a traversable road S t As discussed above with respect to , it may include a state space indicating the passability of roads.

[0256] In step 1506, the technique 1500 obtains a policy from a safety decision component. The safety decision component may be as described above with respect to safety DC 1306 of FIG. 13. Thus, the policy provides actions for road conditions, and the actions constrain the trajectory of the autonomous vehicle along the road. As described above, the state space of the safety decision component includes, for a location of the road, whether motion is detected at that location, and whether an obstacle is detected at that location. The safety decision component includes an action space. The action in the action space to take at a location of the road indicates a movement and a direction of movement.

[0257] In step 1508, the technique 1500 receives an action from the safety decision component. The safety decision component may provide an action at each time step. The action is obtained using the policy. In step 1510, the technique 1500 autonomously traverses the road according to the action. That is, the action may be provided to a trajectory planner of the vehicle, and the vehicle may be controlled according to the action to traverse the road. Autonomously traversing the road according to the action may include identifying an operation scenario along the route and instantiating a decision component for the operation scenario. To illustrate, when the vehicle is traversing the road, an intersection may be identified using sensor data of the vehicle. Then, a DC for traversing the intersection may be instantiated to provide an action to traverse the intersection.

[0258] In one example, the technique 1500 further includes identifying lane segments of the road and storing lane segment information of the lane segments in a navigation map, which may be as described above.

[0259] As used herein, the term "instructions" may include instructions or expressions for performing any method disclosed herein, or any part or parts thereof, and may be implemented in hardware, software, or any combination thereof. For example, instructions may be implemented as information, such as a computer program stored in a memory, that may be executed by a processor to perform any of the methods, algorithms, aspects, or combinations thereof described herein. Instructions or portions thereof may be implemented as a dedicated processor or circuitry, which may include dedicated hardware for performing any of the methods, algorithms, aspects, or combinations thereof described herein. In some implementations, portions of instructions may be distributed across multiple devices or multiple processors on a single device, which may communicate directly or over a network, such as a local area network, a wide area network, the Internet, or combinations thereof.

[0260] As used herein, the terms "example," "embodiment," "implementation," "aspect," "feature," or "element" indicate serving as an example, example, or illustration. Unless expressly stated, any example, embodiment, implementation, aspect, feature, or element is independent of each other example, embodiment, implementation, aspect, feature, or element and may be used in combination with any other example, embodiment, implementation, aspect, feature, or element.

[0261] As used herein, the terms "determining" and "identifying" or any variation thereof includes selecting, ascertaining, calculating, retrieving, receiving, determining, establishing, obtaining, or otherwise identifying or determining in any manner using one or more of the devices shown and described herein.

[0262] As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or" unless otherwise specified or apparent from the context. Furthermore, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless the context clearly indicates a singular reference or unless otherwise specified.

[0263] Furthermore, although for simplicity of description, the drawings and descriptions herein may include a series of steps or stages or sequences, elements of the methods disclosed herein may occur in various orders or simultaneously. Moreover, elements of the methods disclosed herein may occur with other elements not explicitly shown and disclosed herein. Moreover, not all elements of the methods described herein are required to implement a method according to the present disclosure. Although aspects, features, and elements are described herein in certain combinations, each aspect, feature, or element may be used independently or in various combinations with or without other aspects, features, and elements.

[0264] The above aspects, examples and implementations are described to facilitate understanding of the present disclosure and are not intended to be limiting. To the contrary, the present disclosure encompasses various modifications and equivalent arrangements that are within the scope of the appended claims, which should be interpreted broadest possible to encompass all such modifications and equivalent arrangements as are permitted by law.

Claims

1. A method for providing instructions in route planning for an autonomous vehicle, comprising: determining a route based on at least two objectives received from a user, the at least two objectives being prioritized by the user, and if a first of the at least two objectives has a higher priority than a second of the at least two objectives, the second objective is constrained within a margin value of the first objective, the margin value defining a deviation from the first objective to improve the priority of the second objective; receiving a request from the user for a description of an action along the route, the action being to move the autonomous vehicle from a first lane segment to a second lane segment along the route; providing the explanation to the user, the explanation including a descriptor for each of the first objective, the second objective, and a degree of violation of the margin value; A method comprising:

2. The method of claim 1 , wherein the at least two objectives include at least two of a time objective, a comfort objective, an autonomy objective, or an urban area objective.

3. Determining the route based on the at least two objectives includes: maintaining a respective cost of performing a lane segment transition from a current lane segment for at least one of the at least two objectives; The method of claim 1 , comprising:

4. Providing the explanation to the user includes: using the respective costs of executing the lane segment transition from the current lane segment to determine a difference between a first expected discounted reward value associated with a first optimal action for the first objective and a second expected discounted reward value associated with a second optimal action for the first objective, the second action being a selected action and associated with the second objective. The method of claim 3 , comprising:

5. The method of claim 4 , wherein the description includes a descriptor for each of the first action, the second action, the first objective, the second objective, and the degree to which the second action violates the margin value.

6. The method of claim 1 , wherein the instructions are provided in at least one of a visual, tactile, or audio modality.

7. receiving the first objective, the second objective, and the margin value from the user; The method of claim 1 further comprising:

8. The method of claim 7 , wherein the first objective, the second objective, and the margin value are received from the user in a verbal command.

9. An apparatus for route planning for an autonomous vehicle (AV), comprising: receiving from a user a first objective for optimizing a route from a start location to a destination; receiving from the user a second objective for optimizing the route from the starting location to the destination, the first objective and the second objective being prioritized by the user; receiving a margin value from the user defining a deviation from the first objective to improve the priority of the second objective if the first objective has a higher priority than the second objective; determining a route that satisfies the first objective, the second objective, and the margin value; controlling the autonomous vehicle to move from a first lane segment to a second lane segment along the route; receiving a request from the user for an explanation as to why the second lane segment was taken instead of a third lane segment; providing an explanation, the explanation describing the extent to which the second action violates the margin value; 16. An apparatus comprising: a processor configured to:

10. 10. The apparatus of claim 9, wherein determining the route includes obtaining a policy that provides an action to control the autonomous vehicle to move, when the autonomous vehicle is on a first lane segment, into a second lane segment adjacent to the first lane segment.

11. 10. The apparatus of claim 9, wherein the route includes a transition from a first segment of a first lane of a road to a second segment of a second lane of the road along the route.

12. The apparatus of claim 9 , wherein the user provides the first objective, the second objective, and the margin value by verbal commands.

13. The processor, providing a user interface to the user, the user using the user interface to construct a directed graph of objectives including the first objective, the second objective, and the margin value; The apparatus of claim 9 , further configured to:

14. The apparatus of claim 9 , wherein the description includes a descriptor for each of the first objective, the second objective, and the degree to which the second action violates the margin value.

15. 1. A non-transitory computer-readable storage medium comprising executable instructions that, when executed by a processor, facilitate performance of operations for route planning for an autonomous vehicle (AV), the operations comprising: receiving from a user a first objective for optimizing a route from a start location to a destination; receiving from the user a second objective for optimizing the route from the starting location to the destination, the first objective and the second objective being prioritized by the user; receiving a margin value from the user defining a deviation from the first objective to improve the priority of the second objective if the first objective has a higher priority than the second objective; determining a route that satisfies the first objective, the second objective, and the margin value; controlling the autonomous vehicle to move from a first lane segment to a second lane segment along the route; receiving a request from the user for an explanation as to why the second lane segment was taken instead of a third lane segment; the description includes a descriptor for each of the first objective, the second objective, and the degree to which the margin is violated; and Providing said description; 1. A non-transitory computer-readable storage medium comprising:

16. 16. The non-transitory computer-readable storage medium of claim 15, wherein determining the route includes obtaining a policy that provides a second action for controlling the autonomous vehicle to move to the second lane segment adjacent to the first lane segment when the autonomous vehicle is on the first lane segment.

17. 16. The non-transitory computer-readable storage medium of claim 15, wherein the route includes a transition from a first segment of a first lane of a road to a second segment of a second lane of the road along the route.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the user provides the first objective, the second objective, and the margin value by verbal commands.

19. The operation includes: providing a user interface to the user, the user using the user interface to construct a directed graph of objectives including the first objective, the second objective, and the margin value; 20. The non-transitory computer-readable storage medium of claim 15, further comprising an operation for:

Images