Impromptu navigation instructions

Abstract

A computing device (10) may implement a method for providing navigation information about a current trip of a user (502) if the user (502) has not previously initiated a navigation session. The method (800) may include receiving (802) a user (502) request for navigation information about a current itinerary prior to the user (502) initiating a navigation session, and determining (804) a destination for the current itinerary. The method (800) may also include generating (806) one or more sets of navigation guides for going to the destination along one or more routes from a current location of the user (502) based on a current trajectory of the user (502), and providing (808) a response to the request for navigation information based on the generated one or more sets of navigation guides.

Description

Technical Field

[0001] The present disclosure relates generally to navigation systems and, more particularly, to providing responses to impromptu requests for navigation before a user has initiated a navigation session. Background Art

[0002] The background description provided herein is for the purpose of generally presenting the context of the present disclosure. The work of the presently named inventors (to the extent that it is described in this background section) and aspects of the specification that otherwise may not be considered prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art with respect to the present disclosure.

[0003] Today, many users request maps and navigation data for various geographic locations. A software application executed in a computer, smartphone, embedded device, etc. generates step-by-step navigation instructions in response to receiving input from a user specifying a starting point and a destination. The navigation instructions are typically generated for a route that leads the user to the destination in the shortest time. Summary of the invention

[0004] In some scenarios, such as when a user is familiar with a route from the user's starting location (e.g., their home) to a destination (e.g., their place of work), the user does not request navigation directions and does not initiate a navigation session via a navigation application. However, while the user is on the route, they may have questions about the trip. For example, if there is a traffic jam, the user may want to know how long it will take the user to reach the destination. In another example, the user may want to know if an alternate route should be taken to avoid part of the traffic jam.

[0005] In order to receive answers to these questions while the user is on a current trip to a destination without having to initiate a navigation session in the middle of the trip, the impromptu navigation system can receive a request from the user for navigation information about the current trip before initiating the navigation session. For example, the request might be "Hey, Maps, I'm on my way to the office. Should I turn left here?" The impromptu navigation system can determine the destination based on information contained in the request (e.g., "office"). In other scenarios, the impromptu navigation system can determine the destination based on information contained in a previous request, such as a request for navigation directions hours before the user began their trip. In other scenarios, the impromptu navigation system can infer the destination based on user contextual data, such as calendar events, the user's typical destinations on a particular day of the week / time of day, etc.

[0006] Then, based on the determined destination and / or the user's current trajectory, the impromptu navigation system generates navigation directions from the user's current location to the determined destination to provide a response to the request. For example, if the user is traveling east on their current trip, the impromptu navigation system may generate navigation directions to the destination starting from the current location and traveling east. In some implementations, the impromptu navigation system generates multiple sets of navigation directions for comparison to provide a response to the request. Moreover, in some implementations, the impromptu navigation system may use cached navigation directions, such as from a previous request to the destination, that overlap with the user's current trajectory.

[0007] The impromptu navigation system then uses the attributes from the request to analyze one (or more) sets of navigation instructions to provide a response to the request. For example, in response to a user's request as to whether they should turn left, the impromptu navigation system analyzes a set of navigation instructions to the user's office to determine whether the next maneuver in the set of navigation instructions is a left turn. The impromptu navigation system then provides a response to the user indicating whether the next maneuver in the set of navigation instructions is a left turn. For example, the response can be an audio response confirming that the user is correct and should turn left. The response can also be a text or visual response indicating that the user is correct and should turn left. If the user should not turn left, the response can indicate that the user is wrong and can ask the user whether they are willing to receive navigation instructions to the destination. The impromptu navigation system can then initiate a navigation session and provide audio and / or visual navigation instructions to the user.

[0008] In some scenarios, the impromptu navigation system may analyze multiple routes to the user's office to determine which route includes a left turn for an upcoming maneuver. The impromptu navigation system may then provide a response to the user indicating that the next maneuver for one route is a left turn, while the next maneuver for another route is not a left turn. For example, the response may indicate that the next maneuver for the quickest route to the office is a left turn, but if the user prefers a route that is the shortest distance, the user should continue straight.

[0009] By providing responses to requests for navigation information before initiating a navigation session while the user is on a current trip, the user need not initiate a navigation session at the beginning of a route. This can save battery power on the user's client device as well as processing power and bandwidth requirements for at least the portion of the trip for which the user does not request navigation information. Additionally, this can reduce unnecessary distractions to the user when the user is familiar with a large portion of the route by not providing unwanted navigation information, and increase driver safety by, for example, avoiding the driver having to initiate a navigation session mid-trip if the driver is lost or unsure of the next maneuver. Furthermore, the impromptu navigation system further reduces bandwidth requirements by caching navigation directions from previous requests and using the cached navigation directions in some scenarios to respond to requests rather than obtaining navigation directions from a server.

[0010] One exemplary embodiment of the disclosed technology is a method for providing navigation information about a user's current trip without the user previously initiating a navigation session. The method includes receiving a user's request for navigation information about the current trip before the user initiates the navigation session, and determining a destination of the current trip. The method also includes generating one or more sets of navigation instructions for traveling from the user's current location to the destination along one or more routes based on the user's current trajectory, and providing a response to the request for navigation information based on the generated one or more sets of navigation instructions.

[0011] Another exemplary embodiment is a computing device for providing navigation information about a user's current trip without the user having previously initiated a navigation session. The computing device includes one or more processors, and a computer-readable memory, which is optionally non-transitory, coupled to the one or more processors and having instructions stored on the computer-readable memory, which instructions, when executed by the one or more processors, cause the computing device to receive a user's request for navigation information about the current trip before the user initiates the navigation session, and determine a destination for the current trip. The instructions also cause the computing device to generate one or more sets of navigation instructions for traveling from the user's current location to the destination along one or more routes based on the user's current trajectory, and provide a response to the request for navigation information based on the generated one or more sets of navigation instructions.

[0012] Yet another exemplary embodiment is a computer-readable medium, optionally non-transitory, storing instructions for providing navigation information about a user's current trip without the user previously initiating a navigation session, the instructions, when executed by one or more processors, causing the one or more processors to perform the following operations: receiving a user's request for navigation information about the current trip before the user initiates the navigation session, and determining a destination for the current trip. The instructions also cause the one or more processors to generate one or more sets of navigation instructions for traveling from the user's current location to the destination along one or more routes based on the user's current trajectory, and providing a response to the request for navigation information based on the generated one or more sets of navigation instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram of an exemplary communication system in which techniques for providing impromptu navigation information may be implemented;

[0014] Figure 2 depicts an exemplary scenario within a vehicle interior in which a user requests navigation information without previously initiating a navigation session and includes a destination in the request;

[0015] Figure 3 Another exemplary scenario within a vehicle interior is depicted in which a user requests navigation information without previously initiating a navigation session and does not include a destination in the request;

[0016] Figure 4 depicts yet another exemplary scenario within a vehicle interior in which a user requests navigation information without previously initiating a navigation session and does not include a destination in the request;

[0017] Figure 5 Draws in Figure 1 and an exemplary interaction between a user of and a user computing device, wherein the user computing device requests the user to clarify a navigation information request;

[0018] Figure 6 Draws in Figure 1 Another exemplary interaction between a user and a user computing device, wherein the user computing device provides alternative responses to a plurality of potential destinations;

[0019] Figure 7 Draws in Figure 1 and a user computing device, wherein the user computing device provides a prompt for the user to initiate a navigation session to receive navigation directions to the user's destination; and

[0020] Figure 8is a flow chart of an exemplary method for providing navigation information about a user's current trip when the user has not previously initiated a navigation session, the exemplary method being capable of being used in a user's navigation system such as Figure 1 The system is implemented in a computing device such as a user computing device. DETAILED DESCRIPTION

[0021] Overview

[0022] In general, techniques for providing navigation information about a user's current trip when the user has not previously initiated a navigation session can be implemented in one or more user computing devices, one or more network servers, or a system including a combination of these devices. However, for clarity, the following examples focus on an implementation in which a user of a current trip provides a request for navigation information to a user computing device. The user computing device analyzes the request, for example using natural language processing techniques, to determine whether a destination is included in the request. The user computing device can also analyze previous requests and / or user context data, such as calendar events, typical destinations of the user on a particular day of the week / at a particular time of day, etc., to determine the destination of the current trip.

[0023] The user computing device may also determine a current location and / or a current trajectory of the user computing device, for example using sensor data from sensors in the user computing device. For example, the current trajectory may indicate that the user is traveling east on Highway 101. The user computing device may provide the current location, current trajectory, and / or destination to an external server.

[0024] The external server may generate one (or more) sets of navigation instructions from the current location to the destination including the current trajectory. In some implementations, the external server then uses the attributes from the request to analyze the one (or more) sets of navigation instructions to generate a response to the request. The external server may then provide the response to the request to the user computing device. In other implementations, the external server provides the generated one (or more) sets of navigation instructions to the user computing device. The user computing device then analyzes the one (or more) sets of navigation instructions using the attributes from the request to generate a response to the request. The user computing device then presents the response to the user as an audio response and / or a visual response via the user interface of the user computing device.

[0025] Exemplary Hardware and Software Components

[0026] refer to Figure 1, an exemplary communication system 100 in which techniques for providing impromptu navigation information may be implemented includes a user computing device 102. For example, the user computing device 102 may be a portable device, such as a smartphone or a tablet computer. The user computing device 102 may also be a laptop computer, a desktop computer, a personal digital assistant (PDA), a wearable device (such as a smart watch or smart glasses), a virtual reality headset, etc. In some embodiments, the user computing device 102 may be removably mounted in a vehicle, embedded in a vehicle, and / or may be capable of interacting with a head unit of the vehicle to provide navigation instructions.

[0027] The user computing device 102 may include one or more processors 104 and a memory 106 storing machine-readable instructions that can be executed on the processor 104. The processor 104 may include one or more general-purpose processors (e.g., CPUs) and / or special-purpose processing units (e.g., graphics processing units (GPUs)). The memory 106 may optionally be a non-transitory memory and may include one or more suitable memory modules, such as random access memory (RAM), read-only memory (ROM), flash memory, other types of persistent memory, etc. The memory 106 may store instructions for implementing a navigation application 108, which may provide navigation instructions (e.g., by displaying instructions or issuing audio instructions via the user computing device 102), display an interactive digital map, request and receive route data to provide driving, walking or other navigation instructions, and provide various geo-location content (such as traffic, points of interest (POIs), and weather information, etc.). Although the examples described herein for providing impromptu navigation information include driving information, the technology may be applied to navigation information for any suitable mode of transportation, such as walking, cycling, public transportation, etc.

[0028] The navigation application 108 may include an ad-hoc navigation response engine 160. The ad-hoc navigation response engine 160 may receive an audio or text request from the user for navigation information about the user's current trip when the user has not yet initiated a navigation session for the current trip. The user may provide a specific trigger phrase or hotword that causes the ad-hoc navigation response engine 160 to receive an audio request from the user, such as "Hey, maps." The ad-hoc navigation response engine 160 may then analyze the request using natural language processing techniques described below to identify a destination and / or other attributes included in the request.

[0029] When a destination is not included in the request, the ad-hoc navigation response engine 160 may obtain previous navigation requests from the user and may infer the destination based on the destinations in the previous navigation requests. For example, the ad-hoc navigation response engine 160 may obtain previous navigation requests stored at the user computing device 102 or at the external server 120. The ad-hoc navigation response engine 160 may also obtain user context data stored at the user computing device 102 or at the external server 120 to infer the destination, such as calendar data, typical destinations for a particular day of the week / time of day, etc.

[0030] In response to determining or inferring the destination, the impromptu navigation response engine 160 may transmit a request for navigation directions from the user's current location along the current trajectory to the destination to the external server 120. The impromptu navigation response engine 160 may then receive one or more sets of navigation directions for the destination from the external server 120, and may analyze the set(s) of navigation directions to provide a response to the request for navigation information. In other implementations, for example, the impromptu navigation response engine 160 may obtain one or more sets of navigation directions cached at the user computing device 102 and obtained offline from the external server 120 in a previous request.

[0031] In addition, the memory 102 may include a language processing module 109a, which is configured to implement and / or support the technology disclosed in the present invention for providing impromptu navigation information through natural dialogue. That is, the language processing module 109a may include an automatic speech recognition (ASR) engine 109a1, which is configured to transcribe the voice input from the user into multiple groups of text. In addition, the language processing module 109a may include a text-to-speech (TTS) engine 109a2, which is configured to convert text into audio output, such as audio response, audio request, navigation instructions and / or other output of the user. In some scenarios, the language processing module 109a may include a natural language processing (NLP) model 109a3, which is configured to output text transcription, intent interpretation and / or audio output related to the voice input received from the user of the user computing device 102. It should be understood that, as described herein, an ASR engine 109a1 and / or a TTS engine 109a2 may be included as part of the NLP model 109a3 to transcribe user voice input into a set of text, convert text output into audio output, and / or any other suitable functionality described herein as part of a conversation between the user computing device 102 and a user.

[0032] In general, the language processing module 109a may include computer executable instructions for training and operating the NLP model 109a3. In general, the language processing module 109a may train one or more NLP models 109a3 by: establishing a network architecture or topology; and adding layers that may be associated with one or more activation functions (e.g., rectified linear units, softmax, etc.), loss functions, and / or optimization functions. Such training may typically be performed using symbolic methods, machine learning (ML) models, and / or any other suitable training methods. More generally, the language processing module 109a may train the NLP model 109a3 to perform two techniques: grammatical analysis and semantic analysis, which enable the user computing device 102 and / or any other suitable device (e.g., the vehicle computing device 151) to understand the utterances spoken by the user and / or the utterances generated by the text-to-speech program (e.g., TTS engine 109a2) executed by the processor 104.

[0033] Syntactic analysis generally involves analyzing text using basic grammatical rules to identify overall sentence structure, how specific words are organized within a sentence, and how words within a sentence relate to each other. Syntactic analysis may include one or more subtasks, such as tokenization, part of speech (PoS) identification, parsing, lemmatization and stemming, stop word removal, and / or any other suitable subtasks or combinations thereof. For example, using syntactic analysis, the NLP model 109a3 may generate a text transcription based on speech input from a user. Additionally or alternatively, the NLP model 109a3 may receive such a text transcription as a set of text from the ASR engine 109a1 in order to perform semantic analysis on the set of text.

[0034] Semantic analysis generally involves analyzing text in order to understand and / or otherwise capture the meaning of the text. Specifically, the NLP model 109a3 to which semantic analysis is applied can study the meaning of each individual word contained in the text transcription in a process known as lexical semantics. Using these individual meanings, the NLP model 109a3 can examine the various combinations of words contained in the sentences of the text transcription to determine one or more contextual meanings of the words. Semantic analysis can include one or more subtasks, such as word sense disambiguation, relation extraction, sentiment analysis and / or any other suitable subtasks or combinations thereof. For example, using semantic analysis, the NLP model 109a3 can generate one or more intent interpretations based on the text transcription from the grammatical analysis.

[0035] In these aspects, the language processing module 109a may include an artificial intelligence (AI) trained dialog algorithm (e.g., a natural language processing (NLP) model 109a3) that is configured to interact with a user who is accessing the navigation application 108. The user may connect directly to the navigation application 108 to provide verbal input / responses (e.g., voice input), and / or the user request may include text input / responses, which the TTS engine 109a2 (and / or other suitable engines / models / algorithms) may convert into audio input / responses for interpretation by the NLP model 109a3. When the user accesses the navigation application 108, the input / responses spoken by the user and / or generated by the TTS engine 109a2 (or other suitable algorithms) may be analyzed by the NLP model 109a3 to generate text transcriptions and intent interpretations.

[0036] The language processing module 109a can use multiple training speech inputs from multiple users to train one or more NLP models 109a3 to apply these and / or other NLP techniques. Therefore, the NLP model 109a3 can be configured to output a text transcription and an intention interpretation corresponding to the text transcription based on grammatical analysis and semantic analysis of the user's speech input.

[0037] In certain aspects, the language processing module 109a may employ one or more types of machine learning (ML) to train the NLP model 109a3. The ML module 109b, which may store the ML model 109b1, may employ the ML. The ML model 109b1 may be configured to receive a set of text corresponding to a user input and output an intent based on the set of text. The NLP model 109a3 may be and / or include one or more types of ML models, such as the ML model 109b1. More specifically, in these aspects, the NLP model 109a3 may be or include a machine learning model (e.g., a large language model (LLM)) trained by the ML module 109b using one or more text training data sets to output one or more training intents and one or more training destinations, as further described herein. For example, an artificial neural network, a recurrent neural network, a deep learning neural network, a Bayesian model, and / or any other suitable ML model 109b1 may be used to train and / or otherwise implement the NLP model 109a3. In these aspects, training may be performed by iteratively training the NLP model 109a3 using labeled training samples (eg, training user input).

[0038] In the case where the NLP model 109a3 is an artificial neural network, the training of the NLP model 109a3 may produce byproduct weights, or parameters that may be initialized to random values. When iteratively training the network, the weights may be modified by using one of several gradient descent algorithms to reduce losses and converge the values ​​of the network output to an expected value or a "learning" value. In an embodiment, a regression neural network lacking an activation function may be selected, wherein the input data may be normalized by mean centering to determine the accuracy of the loss and quantify the output. Such normalization may use a mean square error loss function and a mean absolute error. Standard techniques such as simple cross validation (hold-out), K-fold cross validation (K-fold) may be used to verify and cross-validate the artificial neural network model. In an embodiment, multiple artificial neural networks may be trained and operated individually, and / or multiple artificial neural networks may be trained and operated jointly individually.

[0039] In an embodiment, one or more NLP models 109a3 may include an artificial neural network with an input layer, one or more hidden layers and an output layer. Each layer in the artificial neural network may include any number of neurons. Multiple layers may link neurons together linearly and may pass output from one neuron to the next neuron, or may be networked together so that neurons transmit input and output in a nonlinear manner. In general, it should be understood that many configurations and / or connections of artificial neural networks are possible. For example, the input layer may correspond to input parameters given as complete sentences, or input parameters separated according to word or character (e.g., fixed width) restrictions. In some embodiments, the input layer may correspond to a large number of input parameters (e.g., one million inputs) and may be analyzed serially or in parallel. In addition, various neurons and / or neuron connections in the artificial neural network may be initialized with any number of weights and / or other training parameters. Each neuron in the hidden layer may analyze one or more input parameters from the input layer, and / or one or more outputs from one or more previous hidden layers to generate decisions or other outputs. The output layer may include one or more outputs, each indicating a prediction. In some embodiments and / or scenarios, the output layer includes only a single output.

[0040] It should be noted that although Figure 1The navigation application 108 is depicted as a standalone application, but the functionality of the navigation application 108 may also be provided in the form of an online service accessible via a web browser executing on the user computing device 102, provided as a plug-in or extension to another software application executing on the user computing device 102, etc. Different versions of the navigation application 108 may generally be provided for different operating systems. For example, the manufacturer of the user computing device 102 may provide a software development kit (SDK) for the Android™ platform that includes the navigation application 108, another SDK for the iOS™ platform, etc.

[0041] The memory 106 may also store an operating system (OS) 110, which may be any type of suitable mobile or general purpose operating system. The user computing device 102 may also include one or more sensors, such as a global positioning system (GPS) 112 or another suitable positioning module, an accelerometer, a gyroscope, a compass, an inertial measurement unit (IMU), etc., a network module 114, a user interface 116 for displaying map data and directions, and an input / output (I / O) module 118. The network module 114 may include one or more communication interfaces, such as hardware, software, and / or firmware for interfaces for communicating via a cellular network, a Wi-Fi network, or any other suitable network such as the network 144 discussed below. The I / O module 118 may include an I / O device capable of receiving input from the surrounding environment and / or a user and providing output to the surrounding environment and / or a user. The I / O module 118 may include a touch screen, a display, a keyboard, a mouse, buttons, keys, a microphone, a speaker, etc. In various implementations, the user computing device 102 may include more than one embodiment of the present invention. Figure 1 Fewer components than those shown, or conversely, additional components may be included.

[0042] The user computing device 102 may communicate with the external server 120 and / or the vehicle computing device 150 via a network 144. The network 144 may include one or more of: an Ethernet-based network, a dedicated network, a cellular network, a local area network (LAN), and / or a wide area network (WAN), such as the Internet. The navigation application 108 may transmit map data, navigation directions, and other geo-located content to the vehicle computing device 150 for display on the instrument display unit 151. In addition, the user computing device 102 may be directly connected to the vehicle computing device 150 via any suitable direct communication link 140, such as a wired connection (e.g., a USB connection).

[0043] In some aspects, the network 144 may include any communication link suitable for short-range communication and may conform to a communication protocol such as, for example, Bluetooth™ (e.g., BLE), Wi-Fi (e.g., Wi-Fi Direct), NFC, ultrasonic signals, etc. Additionally or alternatively, the network 144 may be, for example, Wi-Fi, a cellular communication link (e.g., conforming to 3G, 4G, or 5G standards), etc. In some scenarios, the network 144 may also include a wired connection.

[0044] The external server 120 may be a remotely located server that includes processing capabilities and executable instructions required to perform some / all of the actions described herein with respect to the user computing device 102. For example, the external server 120 may include a language processing module 120a similar to the language processing module 109a included as part of the user computing device 102, and the module 120a may include one or more of an ASR engine 109a1, a TTS engine 109a2, and / or an NLP model 109a3. The external server 120 may also include a navigation application 120b and an ML module 120c, which are similar to the navigation application 108 and the ML module 109b included as part of the user computing device 102.

[0045] The impromptu navigation response engine 160 and the navigation application 120b can operate as components of an impromptu navigation system. Alternatively, the impromptu navigation system can include only server-side components and simply provide responses to requests for navigation information to the impromptu navigation response engine 160. In other words, the impromptu navigation techniques in these embodiments can be implemented transparently to the impromptu navigation response engine 160. As another alternative, the full functionality of the navigation application 120b can be implemented in the impromptu navigation response engine 160.

[0046] Similarly, the language processing modules 109a, 120a may include separate components at the user computing device 102 and the external server 120, may include only server-side components and provide language processing outputs to the impromptu navigation response engine 160, or the entire functionality of the language processing modules 109a, 120a may be implemented at the user computing device 102. Additionally, the ML modules 109b, 120c may include separate components at the user computing device 102 and the external server 120, may include only server-side components and provide ML outputs to the language processing modules 109a, 120a, or the entire functionality of the ML modules 109b, 120c may be implemented at the user computing device 102.

[0047] The vehicle computing device 150 includes one or more processors 152 and a memory 153 storing computer-readable instructions that can be executed by the processor 152. The memory 153 can store a language processing module 153a, a navigation application 153b, and an ML module 153c that are similar to the language processing module 153a, the navigation application 108, and the ML module 109b, respectively. The navigation application 153b can support functionality similar to the navigation application 108 from the vehicle side, and can facilitate the rendering of information displays, as described herein. For example, in some aspects, the user computing device 102 can provide the vehicle computing device 150 with an accepted route that has been accepted by the user, and corresponding navigation instructions provided to the user as part of the accepted route. The navigation application 153b can then proceed to render the navigation instructions within the instrument unit display 151 and / or generate an audio output that verbally provides the navigation instructions to the user via the language processing module 153a.

[0048] In any case, the external server 120 may be communicatively coupled to various databases from which the external server 120 may retrieve navigation-related data, such as a map database 156, a traffic database 157, and a point of interest (POI) database 159. The map database 156 may include map data, such as map tiles, visual maps, road geometry data, road type data, speed limit data, etc. For example, the map database 156 may also include route data for providing navigation instructions, such as driving, walking, biking, or public transportation instructions. The traffic database 157 may store historical traffic information as well as real-time traffic information. The POI database 159 may store descriptions, locations, images, and other information about landmarks or points of interest. Although Figure 1 Databases 156, 157, and 159 are depicted, but external server 120 may be communicatively coupled to additional databases or conversely fewer databases. For example, external server 120 may be communicatively coupled to a database storing weather data.

[0049] Example Request for Navigation Information

[0050] Figure 2 An exemplary scenario 200 is depicted in which a user requests navigation information without previously initiating a navigation session. The user request may be an audio request. More specifically, the user asks, “Hey, Maps, should I take the highway to my office?” 202. The user may be a driver, a front seat passenger, a rear seat passenger, etc. in a vehicle 12. Although Figures 2 to 4 The exemplary scenarios in include requests related to driving directions, but these are just a few examples given for ease of illustration. The improvisational navigation system can be used for any suitable mode of transportation, including driving, walking, cycling or public transportation.

[0051] In addition, although the exemplary scenarios described herein are generally in the context of a voice-based configuration, the impromptu navigation system can also be used in the context of a touch-based or visual interface. For example, a user's request for navigation information can be entered via free-form text input or through a UI element (e.g., a drop-down menu). The response to the user's request can be displayed to the user via the user interface. The embodiments described in the context of a voice-based interface disclosed herein can also be applied to the context of a touch-based interface. All embodiments described in the context of a voice-based interface disclosed herein can be adapted to be applied to the context of a touch-based interface.

[0052] Figure 2 The exemplary vehicle 12 in FIG. 1 includes a client device 10 and a head unit 14. The client device 10 communicates with the head unit 14 of the vehicle 12 via a communication link 16, which may be wired (e.g., Universal Serial Bus (USB)) or wireless (e.g., Bluetooth, Wi-Fi Direct). The client device 10 may also communicate with various content providers, servers, etc. via a wireless communication network, such as a fourth generation or third generation cellular network (4G or 3G, respectively).

[0053] The host unit 14 may include a display 18 for presenting navigation information, such as a digital map. In some implementations, the display 18 is a touch screen and includes a software keyboard for entering text input, which may include the name or address of the destination, the departure point, etc. Hardware input controls 20 and 22 on the host unit 14 and the steering wheel, respectively, may be used to enter alphanumeric characters or perform other functions for requesting navigation guidance. For example, the host unit 14 may also include audio input and output components, such as a microphone 24 and a speaker 26. The speaker 26 may be used to play audio instructions or audio notifications sent from the client device 10.

[0054] In the exemplary scenario 200, the user has not yet initiated a navigation session. In other words, the user has not requested navigation directions from the user's current location to the user's office before starting their current trip to the office. In some implementations, the user may not have yet launched the navigation application 108, but may have just begun interacting with the user computing device 102 by asking, "Hey, Maps, should I take the highway to my office?" The phrase "Hey, Maps" may be a hot word or trigger to activate the navigation application 108. In other implementations, the navigation application 108 may be activated and may continuously or periodically obtain location data of the user computing device 102, for example, to determine the current trajectory of the user computing device 102, but may not be executing a navigation session and / or providing navigation directions to the user's office. In another example, "not initiating a navigation session" may mean that the user computer device 102 is in a state that requires less battery usage and / or less processing power than when a navigation session has been initiated. For example, the user computer device 102 may consume relatively more power when the navigation application 108 has been launched than before the launch.

[0055] In any case, in response to receiving the audio request, the navigation application 108 may communicate with a language processing module 109a, 120a at the user computing device 102 or the external server 120 to interpret the audio request.

[0056] More specifically, the user computing device 102 receives the audio request via an input device (e.g., a microphone as part of the I / O module 118). The user computing device 102 then utilizes the processor 104 to execute instructions included as part of the language processing module 109a to transcribe the audio request into a set of text. The user computing device 102 may cause the processor 104 to execute instructions including, for example, an ASR engine (e.g., ASR engine 109a1) to transcribe the audio request from the speech-based input received by the I / O module 118 into a text transcription of the user input. It should be understood that executing the ASR engine to transcribe the user input into a text transcription may be performed by the user computing device 102, the external server 120, the vehicle computing device 150, and / or any other suitable component or combination thereof.

[0057] This transcription of the audio request may then be analyzed by the processor 104 executing instructions including the language processing module 109a and / or the machine learning module 109b to interpret the text transcription and determine properties of the request, such as a destination. The user computing device 102 may identify the destination by comparing the wording in the audio request to POIs from the POI database 159, addresses contained in the map database 156, or predetermined destinations stored in a user profile, such as "home," "work," "my office," etc.

[0058] In some implementations, the audio request does not include a destination, or the language processing module 109a may identify a wording that corresponds to a destination, but the wording relates to a category of destinations without specifying a particular destination (e.g., "restaurant"). In these scenarios, the impromptu navigation response engine 160 may infer the destination using previous navigation requests with a destination and / or user context data, such as calendar data, typical destinations on a particular day of the week / at a particular time of day, etc. For example, the impromptu navigation response engine 160 may obtain previous navigation requests and / or user context data stored at the user computing device 102 or at an external server 120. The impromptu navigation response engine 160 may then analyze the previous navigation requests and / or user context data in light of the audio request to infer the destination.

[0059] For example, if the audio request includes a destination as "restaurant" but does not specify which restaurant, the ad-hoc navigation response engine 160 may analyze recent navigation requests, set(s) of navigation directions, and / or map data requests that include restaurants as destinations. The ad-hoc navigation response engine 160 may also analyze user contextual data to determine if the user has an upcoming calendar event to go to a particular restaurant, whether the user frequents a particular restaurant based on the time of day / day of the week (e.g., the user always goes to John's Pizza for lunch on Tuesdays), and the like.

[0060] In some implementations, the impromptu navigation response engine 160 may rank multiple restaurants as candidate destinations, such as when multiple restaurants may correspond to "restaurant" based on recent navigation requests and contextual data. For example, the candidate destinations may be ranked according to the recency of the request. More specifically, a navigation request for directions to a first restaurant ten minutes prior to the audio request may result in the first restaurant being ranked higher than a navigation request for directions to a second restaurant an hour prior. The candidate destinations may be scored and / or ranked using any suitable factors (e.g., recency of the search, likelihood of the user visiting the candidate destination at a particular time of day / week, whether the user has a calendar event for the candidate destination, whether the candidate destination is along the user's current trajectory (e.g., the user is traveling south and the candidate destination is south of the user's current location), etc.).

[0061] In some implementations, the impromptu navigation response engine 160 may select the highest ranked or scored candidate destination as the destination, and may infer that the highest ranked or scored candidate destination is the destination mentioned in the audio request. In other implementations, the impromptu navigation response engine 160 may select a candidate destination only if the candidate destination scores above a threshold score or has a probability above a threshold of being the destination mentioned in the audio request. Otherwise, the impromptu navigation response engine 160 may provide a response to the user asking the user to clarify the destination.

[0062] In addition to identifying a destination in a user's request for navigation information, the language processing module 109a may also identify other attributes of the user's request. For example, the language processing module 109a may identify the type of request, such as information about upcoming maneuvers on the current trip, information about the duration of the current trip, traffic information for the current trip, a set of navigation directions for the current trip, etc. The language processing module 109a may also identify specific parameters within the request, such as a specific maneuver in the request (e.g., a left turn, a freeway entrance or exit), the location of a specific maneuver, a specified duration compared to the remaining duration or duration of the maneuver (e.g., "Will I arrive at my destination in 15 minutes?" "Will I still be on the freeway in 20 minutes?", etc.).

[0063] In any case, the impromptu navigation response engine 160 can analyze the destination, attributes and / or parameters contained in or mentioned in the user request to respond to the user's request. More specifically, the impromptu navigation response engine 160 can obtain (e.g., from an external server) a set of navigation directions from the user's current location to the determined or inferred destination.

[0064] In some implementations, the impromptu navigation response engine 160 may determine the user's current trajectory based on the location data of the user computing device 102. For example, the impromptu navigation response engine 160 may continuously or periodically obtain the current location of the user computing device 102 in response to a request from the user or when the navigation application 108 is activated. The impromptu navigation response engine 160 may then determine that the user is traveling in a particular direction on a particular road. The impromptu navigation response engine 160 may then filter out multiple sets of navigation instructions from the user's current location to the destination that do not start traveling in the same direction as the user and / or on the same road as the user. In this way, the impromptu navigation response engine 160 may assume that the user's route to the destination includes the user's current trajectory.

[0065] The impromptu navigation response engine 160 may then analyze the filtered set(s) of navigation directions to generate a response to the user's request for navigation information. In the exemplary scenario 200, the impromptu navigation response engine 160 may determine that the destination in the request 202 is the user's office. The impromptu navigation response engine 160 may obtain the location of the user's office, for example, from a user profile stored at the user computing device 102 or the external server 120. The impromptu navigation response engine 160 may then obtain a set(s) of navigation directions from the user's current location to the user's office. The impromptu navigation response engine 160 may also determine that the user is currently traveling eastward, and may filter out sets of navigation directions that do not begin with traveling eastward.

[0066] The impromptu navigation response engine 160 may also determine, via the language processing modules 109a, 120a, that the request type is information about an upcoming maneuver on the current trip, that the upcoming maneuver is for a freeway entrance, and that the freeway entrance is the nearest freeway that is closest to the destination, i.e., Freeway 90. The impromptu navigation response engine 160 may then analyze the set(s) of navigation instructions to determine if any of the navigation instructions include traveling on Freeway 90. In some implementations, the impromptu navigation response engine 160 may compare a set of navigation instructions that include a route traveling on Freeway 90 to an alternate route that does not include Freeway 90. The impromptu navigation response engine 160 may determine attributes of the route, such as whether the route that includes Freeway 90 is the fastest route, whether the route includes construction, or other attributes of the route.

[0067] Then, the impromptu navigation response engine 160 can generate a response to the user's request, for example, based on the attributes of the route. For example, when the route including Highway 90 is the fastest route, the impromptu navigation response engine 160 can generate a response indicating that the user should take the highway. When there is an alternative route that will allow the user to reach their office faster, the impromptu navigation response engine 160 can generate a response indicating that the user should avoid the highway. In another example, when the route including Highway 90 has a similar duration to the alternative route (e.g., the duration is within one or two minutes of each other, or the duration difference is less than a threshold difference) but the alternative route does not have so much traffic or construction, the impromptu navigation response engine 160 can generate a response indicating that the user should avoid the highway. More generally, the response to the user's request can include one or more sets of navigation instructions to the destination, route information to the destination, traffic information to the destination, the duration of the remaining portion of the current trip to the destination, the duration of a section of the route (e.g., the highway section of the route), traffic information for a section of the route, etc. Before generating the response, the user may not be provided with navigation instructions or driving information (such as described above). Advantageously, the process thus minimizes battery usage and optimizes processing efficiency by only providing navigation directions or driving information as a response to a user request. Such a benefit is provided because the client device 10 or host unit 14, for example, can have its screen / display turned off, be in a device sleep mode, or otherwise be in a device battery optimization mode until a response to a user request is required to be provided as an output from such device.

[0068] In the exemplary scenario 200, the route including Highway 90 is not the fastest route. Therefore, the impromptu navigation response engine 160 generates a response 204 that recommends the user to avoid the highway due to traffic congestion. The impromptu navigation response engine 160 can then provide the response 204 to the user's request for navigation information as an audio output via a speaker, as a visual output on the user interface 116, and / or as a combination of audio / visual output.

[0069] The user computing device 102 may generate the text of the response 204 by utilizing the language processing module 109a and, in some aspects, utilizing a large language model (LLM) (e.g., Language Model for Conversational Applications (LaMDA)) (not shown) included as part of the language processing module 109a. Such an LLM may be tuned / trained to generate the response text based on characteristics of the response and / or the LLM may be trained to receive as input a natural language representation of a response to a request for navigation information and output a set of text representing an audio response based on the characteristics.

[0070] In any case, when the user computing device 102 fully generates the text of the response 204, the device 102 can proceed to synthesize the text into speech for audio output to the user's request. Specifically, the user computing device 102 can transmit the text of the response 204 to a TTS engine (e.g., TTS engine 109a2) so that the response 204 can be audibly output through a speaker (e.g., speaker 26) so that the user can hear and interpret the response. In addition or alternatively, the user computing device 102 can also visually prompt the user by displaying the text of the response on a display screen (e.g., instrument display unit 151, user interface 116) so that the user can interact with the display screen (e.g., click, tap, slide, etc.) and / or verbally confirm the response 204.

[0071] Figure 3 Another exemplary scenario 300 is depicted in which a user requests navigation information without previously initiating a navigation session. In this scenario 300, the user does not include a destination in the request. More specifically, the user asks, "Hey, Maps, which exit should I take to exit this roundabout?" 302. As in exemplary scenario 200, the user did not previously initiate a navigation session prior to providing this request for navigation information.

[0072] In scenario 300, the user request does not include a destination. Therefore, the impromptu navigation response engine 160 may infer the destination using previous navigation requests with a destination and / or user contextual data, such as calendar data, typical destinations on a particular day of the week / time of day, etc. For example, the impromptu navigation response engine 160 may determine that the user's most recent navigation request was for navigation directions to John's Pizza. In another example, the impromptu navigation response engine 160 may determine that the user's calendar includes a meeting at John's Pizza within a threshold duration (e.g., 15 minutes, 30 minutes, one hour, etc.) of the current time. Therefore, the impromptu navigation response engine 160 may infer that the destination is John's Pizza.

[0073] In addition to inferring the destination, the language processing module 109a can also identify other attributes of the user's request. For example, the language processing module 109a can identify the type of request as information about an upcoming maneuver on the current trip. The language processing module 109a can also identify specific parameters in the request, such as the roundabout exit for the next maneuver on the current trip.

[0074] The impromptu navigation response engine 160 may then obtain (e.g., from an external server) a set(s) of navigation directions from the user's current location to the inferred destination (John's Pizza). The impromptu navigation response engine 160 may also determine the user's current trajectory based on the location data of the user computing device 102. For example, when the navigation application 108 is activated, or in any other suitable situation, such as when the user computing device 102 detects that the user is traveling (e.g., the speed of the user computing device 102 is above a threshold speed, such as 10 miles per hour), the impromptu navigation response engine 160 may continuously or periodically obtain the current location of the user computing device 102 in response to a request from the user. The impromptu navigation response engine 160 may then determine that the user is traveling in a particular direction on a particular road.

[0075] In addition, the impromptu navigation response engine 160 can determine the user's current trajectory based on the user's request for navigation information. More specifically, the request indicates that the user is traveling on a roundabout, so the impromptu navigation response engine 160 can determine that the particular road the user is traveling on is a roundabout. The impromptu navigation response engine 160 can compare the trajectory obtained from the location data with the trajectory information indicated in the request (e.g., a roundabout) to ensure that the trajectory obtained from the location data matches the trajectory information indicated in the request. If there is a mismatch, the impromptu navigation response engine 160 can adjust the trajectory obtained from the location data to the nearest road that matches the trajectory information indicated in the request, or can provide a response to the user to request clarification regarding the request.

[0076] In any case, the impromptu navigation response engine 160 can filter out multiple sets of navigation instructions from the user's current location to the destination that do not include a roundabout and / or do not include leaving the roundabout as the next maneuver. The impromptu navigation response engine 160 can then analyze the filtered set(s) of navigation instructions to generate a response to the user's request for navigation information. More specifically, the impromptu navigation response engine 160 can analyze the filtered set(s) of navigation instructions (each of which includes leaving the roundabout) to determine the roundabout exit for going to John's Pizza. In some scenarios, the impromptu navigation response engine 160 can identify the same roundabout exit (e.g., the third roundabout exit) for each filtered set(s) of navigation instructions, and can determine that this is the exit for going to John's Pizza.

[0077] In other scenarios, the impromptu navigation response engine 160 can identify different roundabout exits for different groups of navigation instructions. In these scenarios, the impromptu navigation response engine 160 can identify the roundabout exit for the fastest route to John's Pizza, the roundabout exit for the shortest route to John's Pizza, the roundabout exit for a route that avoids traffic or construction, etc. The impromptu navigation response engine 160 can then provide a response 304 to the user's request for navigation information as an audio output via a speaker, as a visual output on the user interface 116, and / or as a combination of audio / visual outputs. The response 304 can indicate the destination (e.g., to go to the restaurant, you need to take the third exit on the roundabout) so that the user can confirm that the inferred destination is the correct destination. If the inferred destination is not the correct destination, the user can provide a subsequent request.

[0078] Figure 4 Yet another exemplary scenario 400 is depicted in which a user requests navigation information without previously initiating a navigation session. In this scenario 400, the user also does not include a destination in the request. More specifically, the user asks, "Hey, maps, how long?" 402. As in the exemplary scenarios 200, 300, the user did not previously initiate a navigation session prior to providing this request for navigation information.

[0079] In scenario 400, the user request does not include a destination. Therefore, the impromptu navigation response engine 160 may infer the destination using previous navigation requests with a destination and / or user contextual data, such as calendar data, typical destinations for a particular day of the week / time of the day, etc. For example, the impromptu navigation response engine 160 may determine that at the current time and / or day of the week (e.g., Monday at 5 p.m.), the user typically returns home from get off work. Therefore, the impromptu navigation response engine 160 may infer that the destination is the user's home.

[0080] In addition to inferring the destination, the language processing module 109a can also identify other attributes of the user's request. For example, the language processing module 109a can identify the type of request as information about the duration of the current trip. The language processing module 109a can also identify specific parameters within the request, such as the amount of time remaining to reach the destination.

[0081] The impromptu navigation response engine 160 may then obtain (e.g., from an external server) a set(s) of navigation directions from the user's current location to the inferred destination (the user's home). The impromptu navigation response engine 160 may also determine the user's current trajectory based on the location data of the user computing device 102. For example, the impromptu navigation response engine 160 may continuously or periodically obtain the current location of the user computing device 102 in response to a request from the user or when the navigation application 108 is activated. The impromptu navigation response engine 160 may then determine that the user is traveling in a particular direction on a particular road.

[0082] The impromptu navigation response engine 160 may then filter out multiple sets of navigation instructions that do not start traveling in the same direction and on the same road as the user from the user's current location to the destination. The impromptu navigation response engine 160 may then analyze the filtered set(s) of navigation instructions to generate a response to the user's request for navigation information. More specifically, the impromptu navigation response engine 160 may analyze the filtered set(s) of navigation instructions and / or current traffic data on the corresponding routes to identify the duration of each set of navigation instructions. The impromptu navigation response engine 160 may then select the duration of the fastest route as the duration included in the response 404.

[0083] In other implementations, the impromptu navigation response engine 160 may select the duration of the most commonly used route as the duration included in the response 404. More specifically, the fastest route may include several side roads and / or shortcuts that most users are not typically aware of. Another route may include a road that the user is more likely to take, such as a highway, and the impromptu navigation response engine 160 may select the duration of this route as the most likely duration for the user. In another example, the most commonly used route may be the route that the user typically takes based on the user's navigation history. The impromptu navigation response engine 160 may compare the filtered set(s) of navigation instructions with the route from the user's workplace to the user's home to identify a set(s) of navigation instructions in the filtered set(s) of navigation instructions that matches the route that the user typically takes from work to their home.

[0084] The impromptu navigation response engine 160 may then provide a response 404 to the user's request for navigation information as an audio output via a speaker, as a visual output on the user interface 116, and / or as a combination of audio / visual output. The response 404 may include the selected duration and may indicate that the user will arrive at their destination in approximately 30 minutes.

[0085] Figures 5 to 7 An exemplary interaction between a user 502 and a user computing device 102 is depicted when the user requests navigation information prior to initiating a navigation session. Figure 5 In the exemplary scenario 500 of FIG. 5 , user 502 does not include a destination in the request. More specifically, user 502 asks 504 a, “Hey, Maps, should I get off at the next stop?”

[0086] The ad-hoc navigation response engine 160 may then infer a destination using previous navigation requests with a destination and / or user contextual data, such as calendar data, typical destinations at a particular day of the week / time of the day, etc. The ad-hoc navigation response engine 160 may identify multiple candidate destinations (e.g., the user's office and John's Restaurant) and may rank or score the candidate destinations using any suitable factors (e.g., recency of the search, likelihood of the user visiting the candidate destination at a particular time of the day / day of the week, whether the user has a calendar event for the candidate destination, etc.). In this scenario 500, neither candidate destination has a score above a threshold score, a likelihood of being a destination mentioned in the request above a threshold likelihood, and / or a score that is higher than the score of another candidate destination by more than a threshold amount.

[0087] Thus, the user computing device 102 may transmit 504b a first response 506 to the user 502, prompting the user 502 to clarify whether the destination is John's Restaurant or the user's office. The user 502 may then respond by transmitting 504c a clarifying response indicating that the destination is the user's office. The user computing device 102 may then determine the user's current trajectory based on the location data of the user computing device 102 and / or trajectory information included in the user's request for navigation information (e.g., taking a bus). The user computing device 102 may also obtain one or more sets of navigation directions from the user's current location to the user's office, and may filter out sets of navigation directions that do not include taking a bus from the user's current location.

[0088] The user computing device 102 may then analyze the filtered set(s) of navigation directions to generate a second response to the user's request for navigation information 508. In some scenarios, the user computing device 102 may identify the same bus stop (e.g., Willoughby as the next stop) for each set of navigation directions in the filtered set(s) of navigation directions and may determine that this is the bus stop to the user's office.

[0089] In other scenarios, the user computing device 102 may identify different bus stops for different sets of navigation directions. In these scenarios, the user computing device 102 may identify a bus stop for the quickest route to the user's office, a bus stop for the shortest route to the user's office, a bus stop for a route that avoids traffic or construction, etc. The user computing device 102 may then provide a second response 508 to the user's request for navigation information as an audio output via a speaker, as a visual output on the user interface 116, and / or as a combination of audio / visual output.

[0090] It should be noted that the user computing device 102 may generally allow the user 502 several seconds (e.g., 5 to 10 seconds) to respond after the first response 506 including the audio request is transmitted through the speaker 26, so as to give the user 502 sufficient time to think of an appropriate response without continuously listening to the interior of the vehicle. By default, the user computing device 102 may not activate the microphone and / or other listening devices (e.g., included as part of the I / O module 118) while running the navigation application 108, and / or while information received through the microphone is processed by or in accordance with, for example, the processor 104, the language processing module 109a, the machine learning module 109b, and / or the OS 110. Therefore, the user computing device 102 may not actively listen to the interior of the vehicle during the navigation session and / or at any other time, except when the user computing device 102 provides the user 502 with an audio request, the user computing device 102 may expect a verbal response to the audio request from the user 502 within a few seconds of the transmission.

[0091] Figure 6 Draws the Figure 5 . In exemplary scenario 600, user 502 asks 604 “Hey, Maps, should I get off at the next stop?” This is the same user request as in exemplary scenario 500. Therefore, user computing device 102 identifies multiple candidate destinations (e.g., the user's office and John's Restaurant) for the user request, wherein none of the candidate destinations has a score above a threshold score, a likelihood of being the destination mentioned in the request above a threshold likelihood, and / or a score that is higher than a threshold amount than the score of another candidate destination.

[0092] However, rather than prompting the user 502 to clarify whether the destination is John's or the user's office, the user computing device 102 generates a response 606 that provides navigation information to two candidate destinations. More specifically, the response 606 states, "To get to your office, get off at the next stop, Willoughby. To get to John's, get off at Main Street, three stops away." The user 502 can then determine the stop based on the destination of the user 502. Rather than selecting one of the candidate destinations as the destination of the response 606 or prompting the user 502 to clarify the destination, the user computing device 102 can include an alternative response for each candidate destination in the response 606.

[0093] Figure 7 Another exemplary scenario 700 is depicted in which a user computing device prompts a user 502 to initiate a navigation session in response to a user request for navigation information. In the exemplary scenario 700, the user asks 704 "Hey, Maps, how long does it take to get to Bill's house?"

[0094] The user computing device 102 may identify the address of Bill's home based on contact information stored for Bill at the user computing device 102, based on a previous navigation request to Bill's home, based on a pre-stored address of Bill's home, etc. The user computing device 102 may then obtain a set(s) of navigation directions from the user's current location to Bill's home.

[0095] The user computing device 102 may also determine the current trajectory of the user 502 based on the location data of the user computing device 102. For example, the user computing device 102 may determine that the user 502 is currently traveling south on Highway 90. The user computing device 102 may then analyze a set(s) of navigation instructions in light of the current trajectory of the user 502 to generate a response to the user 502's request for navigation information. In the exemplary scenario 700, the user computing device 102 may determine that the fastest route to Bill's house by continuing along the highway will take approximately 30 minutes. However, the user computing device 102 may also determine that there is a faster route if the user 502 leaves Highway 90 and travels to Bill's house along other roads.

[0096] Thus, the user computing device 102 may generate a response 706 to the user 502's request indicating that the expected duration of driving to Bill's house on the highway is approximately 30 minutes. However, if the user 502 leaves the highway, there is a faster route 726. The response 706 may be presented as an audio response 706, and may also be presented as a text response 724 on the user interface 116 of the user computing device 102. Moreover, the text response 724 and / or the audio response 706 may include a prompt with user controls 724a, 724b for the user 502 to select whether they want to receive turn-by-turn directions for the faster route 726. In response to receiving a selection of the user control 724a indicating that the user 502 wants to initiate a navigation session (e.g., via a touch selection or audio input), the user computing device 102 may initiate a navigation session and provide navigation directions for the faster route to Bill's house as audio directions and / or via a navigation display 722 on the user computing device 102.

[0097] Exemplary method for providing impromptu navigation information

[0098] Figure 8 is a flow chart of an exemplary method 800 for providing navigation information about a user's current trip when the user has not previously initiated a navigation session, which method may be used in a user's computer system such as Figure 1 It will be understood that in Figure 8 Throughout the description of FIG. 1 , in some implementations, actions described as being performed by the user computing device 102 may be performed by the external server 120, the vehicle computing device 150, and / or may be performed in parallel by the user computing device 102, the external server 120, and / or the vehicle computing device 150. For example, the user computing device 102, the external server 120, and / or the vehicle computing device 150 may utilize the language processing modules 109a, 120a, 153a and / or the machine learning modules 109b, 120c, 153c to provide impromptu navigation information.

[0099] The method 800 may be implemented in a set of instructions stored on a computer-readable memory and executable at one or more processors (eg, processor 104) of the user computing device 102. For example, the method 800 may be performed by the impromptu navigation response engine 160.

[0100] At block 802, the impromptu navigation response engine 160 may receive a request for navigation information about a current trip before a user initiates a navigation session. The request may be received as an audio request or via text input. The user may provide a specific trigger phrase or hotword, such as "Hey, maps," that causes the impromptu navigation response engine 160 to receive an audio request from the user. In some implementations, the user may not have launched the navigation application 108 before providing the request. In other implementations, the navigation application 108 may be activated and may continuously or periodically obtain location data for the user computing device 102, for example, to determine the current trajectory of the user computing device 102, but may not be executing a navigation session and / or providing navigation directions to a destination.

[0101] The ad-hoc navigation response engine 160 may then determine the destination of the user's current trip at block 804. For example, the ad-hoc navigation response engine 160 may identify the destination by comparing the wording in the request for navigation information to a POI from the POI database 159, an address contained in the map database 156, or a predetermined destination stored in the user profile (such as "home," "work," "my office," etc.).

[0102] In some implementations, the request does not include a destination, or the language processing module 109a may identify a wording that corresponds to a destination, but the wording relates to a category of destinations without specifying a particular destination (e.g., "restaurant"). In these scenarios, the ad-hoc navigation response engine 160 may infer the destination using previous navigation requests with destinations and / or user contextual data, such as calendar events, typical destinations on a particular day of the week / time of day, etc.

[0103] In some implementations, such as when multiple destinations may correspond to the destination based on recent navigation requests and contextual data, the impromptu navigation response engine 160 may rank the candidate destinations. Candidate destinations may be scored and / or ranked using any suitable factors (e.g., recency of the search, likelihood of the user visiting the candidate destination at a particular time of day / week, whether the user has a calendar event for the candidate destination, etc.).

[0104] In some implementations, the impromptu navigation response engine 160 may select the highest ranked or scored candidate destination as the destination, and may infer that the highest ranked or scored candidate destination is the destination mentioned in the audio request. In other implementations, the impromptu navigation response engine 160 may select a candidate destination only if the candidate destination scores above a threshold score or has a probability above a threshold of being the destination mentioned in the audio request. Otherwise, the impromptu navigation response engine 160 may provide a response to the user asking the user to clarify the destination. In other implementations, the impromptu navigation response engine 160 may select multiple destinations and may include alternative group navigation information for each destination in the response.

[0105] At block 806, the impromptu navigation response engine 160 may generate one (or more) sets of navigation instructions for traveling from the user's current location along a route to the destination based on the user's current trajectory. For example, the impromptu navigation response engine 160 may communicate with the external server 120 to receive one (or more) sets of navigation instructions from the map database 156. In other implementations, the impromptu navigation response engine 160 may obtain offline navigation instructions cached at the user computing device 102, for example, after receiving navigation instructions from a previous request for a route to the destination. The impromptu navigation response engine 160 may determine the user's current trajectory based on the location data of the user computing device 102. For example, the impromptu navigation response engine 160 may continuously or periodically obtain the current location of the user computing device 102 in response to a request from the user or when the navigation application 108 is activated. Then, the impromptu navigation response engine 160 may determine that the user is traveling in a specific direction on a specific road. The impromptu navigation response engine 160 may filter out multiple sets of navigation instructions from the user's current location to the destination that do not start traveling in the same direction as the user and on the same road as the user.

[0106] Then at block 808, the impromptu navigation response engine 160 may provide a response to the request for navigation information based on the generated set(s) of navigation directions. The impromptu navigation response engine 160 may generate text for the response by utilizing the language processing module 109a and, in some aspects, utilizing an LLM (e.g., LaMDA) included as part of the language processing module 109a. Such an LLM may be tuned / trained to generate response text based on characteristics of the response, and / or the LLM may be trained to receive as input a natural language representation of a response to a request for navigation information and output a set of text representing an audio response based on the characteristics.

[0107] In any case, when the impromptu navigation response engine 160 fully generates the text of the response, the user computing device 102 can proceed to synthesize the text into speech for audio output of the user's request. Specifically, the user computing device 102 can transmit the text of the response to a TTS engine (e.g., TTS engine 109a2) so that the response can be audibly output through a speaker (e.g., speaker 206) so that the user can hear and interpret the response. In addition or alternatively, the user computing device 102 can also visually prompt the user by displaying the text of the response on a display screen (e.g., instrument display unit 151, user interface 116) so that the user can interact with the display screen (e.g., click, tap, slide, etc.) and / or verbally confirm the response.

[0108] Additional Considerations

[0109] The following additional considerations apply to the foregoing discussion. Throughout the specification, multiple instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are depicted and described as separate operations, one or more of the individual operations may be performed simultaneously, and it is not required to perform the operations in the order depicted. The structures and functionality presented as separate components in the exemplary configuration may be implemented as combined structures or components. Similarly, the structures and functionality presented as separate components may be implemented as separate components. These and other variations, modifications, additions, and improvements all fall within the scope of the subject matter of the present disclosure.

[0110] In addition, some embodiments are described herein as including logic or a number of components, modules or mechanisms. A module may constitute a software module (e.g., a code stored on a machine-readable medium) or a hardware module. A hardware module is a tangible unit that can perform certain operations and can be configured or arranged in a particular manner. In an exemplary embodiment, one or more computer systems (e.g., an independent computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or an application portion) to operate as a hardware module that performs certain operations as described herein.

[0111] In various embodiments, hardware modules can be implemented mechanically or electronically. For example, a hardware module may include dedicated circuits or logic that are permanently configured to perform certain operations (e.g., as a dedicated processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)). A hardware module may also include programmable logic or circuits (e.g., contained within a general-purpose processor or other programmable processor) that are temporarily configured by software to perform certain operations. It should be understood that the decision to implement a hardware module mechanically, in a dedicated and permanently configured circuit, or in a temporarily configured circuit (e.g., configured by software) may be driven by cost and time considerations.

[0112] Thus, the term "hardware" should be understood to encompass tangible entities, i.e., physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) entities to operate in a particular manner or perform certain operations described herein. As used herein, a "hardware-implemented module" refers to a hardware module. Given implementations in which hardware modules are temporarily configured (e.g., programmed), each hardware module need not be configured or instantiated at any one time. For example, where a hardware module includes a general-purpose processor configured using software, the general-purpose processor can be configured as correspondingly different hardware modules at different times. The software can configure the processor accordingly, for example, to constitute a particular hardware module at one time and to constitute a different hardware module at a different time.

[0113] Hardware modules can provide information to other hardware, and receive information from other hardware. Therefore, the described hardware modules can be regarded as communication coupled. In the case of multiple such hardware modules being present at the same time, communication can be achieved by signal transmission (e.g., via appropriate circuits and buses) connecting the hardware modules. In the embodiment where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, by storing and retrieving information in a memory structure to which multiple hardware modules have access rights. For example, a hardware module can perform an operation and store the output of the operation in a memory device coupled with its communication. Then, another hardware module can access the memory device later to retrieve and process the stored output. The hardware module can also initiate communication with an input or output device, and can operate on resources (e.g., information sets).

[0114] The method 800 may include one or more functional blocks, modules, separate functions or routines in the form of tangible computer executable instructions stored in a computer readable storage medium, optionally in a non-transitory computer readable storage medium, and executed using a processor of a computing device (e.g., a server device, a personal computer, a smart phone, a tablet computer, a smart watch, a mobile computing device, or other client computing device, as described herein). The method 800 may be included as, for example, any back-end server (e.g., a map data server, a navigation server, or any other type of server computing device, as described herein), part of a client computing device module of the exemplary environment, or as part of a module external to such an environment. Although the figures may be described with reference to other figures for ease of explanation, the method 800 may be utilized with other objects and user interfaces. In addition, although the above explanation describes the steps of the method 800 as being performed by a specific device (such as a user computing device), this is done for illustrative purposes only. The blocks of the method 800 may be performed by one or more devices or other parts of the environment.

[0115] The various operations of the exemplary methods described herein may be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute a processor-implemented module that operates to perform one or more operations or functions. In some exemplary embodiments, the modules mentioned herein may include processor-implemented modules.

[0116] Similarly, the method or routine described herein can be implemented by a processor at least in part. For example, at least some operations of the method can be performed by a hardware module implemented by one or more processors or processors. The execution of some operations can be distributed between one or more processors that not only reside in a single machine but also are deployed on many machines. In some exemplary embodiments, one or more processors can be located at a single location (for example, in a home environment, an office environment or as a server group), and in other embodiments, the processor can be distributed in many locations.

[0117] The one or more processors may also operate to support execution of related operations in a "cloud computing" environment or as a SaaS. For example, as indicated above, at least some operations may be performed by a set of computers (as an example of a machine including a processor) that are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs).

[0118] Furthermore, the drawings depict some embodiments of the exemplary environment for illustrative purposes only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods depicted herein may be employed without departing from the principles described herein.

[0119] After reading this disclosure, those skilled in the art will appreciate additional alternative structural and functional designs for determining locations and routes through natural dialogue through the principles disclosed herein. Therefore, although specific embodiments and applications have been illustrated and described, it should be understood that the disclosed embodiments are not limited to the precise constructions and components disclosed herein. Various modifications, changes and variations that will be apparent to those skilled in the art may be made to the arrangement, operation and details of the methods and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims.

Claims

1. A method for providing navigation information about a user's current journey without the user having previously initiated a navigation session, the method comprising: receiving, at one or more processors, a user request for navigation information related to a current trip prior to the user initiating a navigation session; determining, by the one or more processors, a destination of the current trip; generating, by the one or more processors, one or more sets of navigation directions for traveling from the user's current location to the destination along one or more routes based on the user's current trajectory; as well as A response to the request for navigation information is provided by the one or more processors based on the generated one or more sets of navigation directions. 2 . The method of claim 1 , wherein the request for navigation information is received before launching a navigation application.

3. The method of claim 1 or 2, wherein the request does not include the destination of the current trip.

4. The method of any preceding claim, wherein determining the destination of the current trip comprises at least one of: determining, by the one or more processors, the destination of the current trip as the destination included in the request for navigation information; inferring, by the one or more processors, the destination of the current trip based on destinations included in previous requests; or The destination of the current trip is inferred by the one or more processors based on user context data.

5. A method as claimed in any preceding claim, wherein providing a response to the request for navigation information comprises providing at least one of: The generated one or more sets of navigation instructions, route information to the stated destination, Transportation information to the stated destination, or The duration of the remainder of the current trip to the destination.

6. A method as claimed in any preceding claim, wherein the destination is a destination included in a previous request, and wherein generating the one or more sets of navigation directions comprises: The one or more sets of navigation directions cached from the previous request are obtained by the one or more processors.

7. A method as claimed in any preceding claim, wherein the request is an audio request and the response to the request is an audio response.

8. A method as claimed in any preceding claim, wherein the request is for the duration of a remaining portion of the current trip to a particular destination, and the response comprises an indication of the duration of the remaining portion of the current trip to the particular destination based on a most likely route to the particular destination according to the current trajectory and current traffic data on the route.

9. A computing device for providing navigation information about a user's current journey without the user having previously initiated a navigation session, the computing device comprising: one or more processors; as well as a computer-readable memory coupled to the one or more processors and having stored thereon instructions that, when executed by the one or more processors, cause the computing device to: receiving a user request for navigation information about a current trip before the user initiates a navigation session; determining a destination of the current trip; generating one or more sets of navigation directions for traveling from the user's current location to the destination along one or more routes based on the user's current trajectory; as well as A response to the request for navigation information is provided based on the generated one or more sets of navigation directions.

10. The computing device of claim 9, wherein the request for navigation information is received prior to launching a navigation application.

11. The computing device of claim 9 or 10, wherein the request does not include the destination of the current trip.

12. The computing device according to any one of claims 9 to 11, wherein: To determine the destination of the current trip, the instructions cause the computing device to perform at least one of the following operations: determining the destination of the current trip as the destination included in the request for navigation information; inferring the destination of the current trip based on destinations included in previous requests; or The destination of the current trip is inferred based on user context data.

13. The computing device according to any one of claims 9 to 12, wherein: To provide a response to the request for navigation information, the instructions cause the computing device to provide at least one of: The generated one or more sets of navigation instructions, route information to the stated destination, Transportation information to the stated destination, or The duration of the remainder of the current trip to the destination.

14. A computing device as claimed in any one of claims 9 to 13, wherein the destination is a destination included in a previous request, and wherein, To generate the one or more sets of navigation directions, the instructions cause the computing device to: The one or more sets of navigation directions cached from the previous request are obtained.

15. The computing device of any one of claims 9 to 14, wherein the request is an audio request and the response to the request is an audio response.

16. A computing device as claimed in any one of claims 9 to 15, wherein the request is for the duration of the remaining portion of the current trip to a specific destination, and the response includes an indication of the duration of the remaining portion of the current trip to the specific destination based on the most likely route to the specific destination based on the current trajectory and current traffic data on the route.

17. A computer-readable medium storing instructions for providing navigation information about a user's current trip when the user has not previously initiated a navigation session, the instructions, when executed by one or more processors, causing the one or more processors to: receiving a user request for navigation information about a current trip before the user initiates a navigation session; determining a destination of the current trip; generating one or more sets of navigation directions for traveling from the user's current location to the destination along one or more routes based on the user's current trajectory; as well as A response to the request for navigation information is provided based on the generated one or more sets of navigation directions.

18. The computer-readable medium of claim 17, wherein the request for navigation information is received prior to launching a navigation application.

19. The computer-readable medium of claim 17 or 18, wherein the request does not include the destination of the current trip.

20. The computer readable medium of any one of claims 17 to 19, wherein: To determine the destination of the current trip, the instructions cause the one or more processors to perform at least one of the following operations: determining the destination of the current trip as the destination included in the request for navigation information; inferring the destination of the current trip based on destinations included in previous requests; or The destination of the current trip is inferred based on user context data.

Images