Ad hoc navigation instruction

Abstract

The computing device (10) may implement a method for providing navigation information regarding the current itinerary of the user (502) without the user (502) having previously initiated a navigation session. The method (800) may include receiving a navigation information request regarding the current itinerary of the user (502) (802) and identifying a destination of the current itinerary (804) prior to the user (502) starting a navigation session. The method (800) may also include generating one or more navigation guidance sets for moving along one or more routes from the current location of the user (502) to the destination based on the current trajectory of the user (502) (806) and providing a response to the navigation information request based on the one or more generated navigation guidance sets (808).

Description

Technical Field

[0001] The present disclosure generally relates to a navigation system, and more specifically to providing a response to an ad-hoc request regarding navigation while a user has not started a navigation session.

Background Art

[0002] The description of the background art provided herein is for the purpose of generally presenting the context of the present disclosure. Within the scope described in this background art section, the research of the inventors named herein, as well as aspects of the description that could not be regarded as prior art at the time of filing, are not admitted as prior art to the present disclosure, either explicitly or implicitly.

[0003] Currently, many users request maps and navigation data of various geographical locations. Software applications running on computers, smartphones, embedded devices, etc. generate step-by-step navigation guidance in response to receiving input from a user specifying a starting point and a destination. Navigation guidance is typically generated with respect to a route that guides the user to the destination in the shortest time.

Summary of the Invention

[0004] In some scenarios, such as when the user is familiar with the route from the starting position of the user (e.g., the user's home) to the destination (e.g., the user's workplace), the user does not request navigation guidance and does not start a navigation session via the navigation application. However, while the user is on the route, the user may have questions about the journey. For example, in the presence of traffic, the user may want to know how much time it will take to reach the destination. In other examples, the user may want to know whether it is necessary to take an alternative route to avoid some of the traffic.

[0005] When a user is moving to a destination on the current journey, in order to receive answers to these questions without the need to start a navigation session during the journey, the ad-hoc navigation system can receive from the user a navigation information request regarding the current journey prior to the start of the navigation session. For example, the request can be something like "Hey, Map, I'm on my way to the office. Should I turn left here?" Based on the information included in the request (e.g., "office"), the ad-hoc navigation system can identify the destination. In other scenarios, the ad-hoc navigation system can identify the destination based on information included in previous requests, such as a navigation guidance request several hours before the user starts the journey. In still other scenarios, the ad-hoc navigation system can infer the destination based on user context data, such as calendar events, the user's usual destination at a specific day / time, etc.

[0006] Next, based on the identified destination and / or the user's current trajectory, the ad-hoc navigation system generates navigation guidance from the user's current location to the identified destination to provide a response to the request. For example, if the user is moving east on the current journey, the ad-hoc navigation system can generate navigation guidance to the destination starting from moving east from the current location. In some embodiments, the ad-hoc navigation system generates multiple sets of navigation guidance for comparison to provide a response to the request. Also, in some embodiments, the ad-hoc navigation system can use cached navigation guidance that overlaps with the user's current trajectory, for example, from a previous request to the destination.

[0007] Next, the ad-hoc navigation system analyzes the navigation guidance set(s) using the attributes from the request to provide a response to the request. For example, in response to a user request regarding whether to turn left, the ad-hoc navigation system analyzes the navigation guidance set to the user's office to determine whether the next maneuver in the set is a left turn. Next, the ad-hoc navigation system provides the user with a response indicating whether the next maneuver in the set is a left turn. For example, the response can be an audio response that confirms to the user that they are correct and should turn left. The response can also be a text or visual response indicating to the user that they are correct and should turn left. If the user should not turn left, the response can indicate to the user that they are incorrect and can ask the user if they want to receive navigation guidance to the destination. Next, the ad-hoc navigation system can start a navigation session and provide the user with audio and / or visual navigation instructions.

[0008] In some scenarios, the ad-hoc navigation system can analyze multiple routes to the user's office to identify which route includes a left turn in the next maneuver. Next, the ad-hoc navigation system can provide the user with a response indicating that the next maneuver of one route is a left turn while the next maneuver of another route is not. For example, the response can indicate that the next maneuver of the fastest route to the office is a left turn, but if the user prefers the shortest distance route, they should continue straight.

[0009] While the user is moving in the current journey, by providing a response to a navigation information request prior to the start of the navigation session, the user does not need to start the navigation session at the start of the route. This saves the battery power of the user's client device, as well as the processing power and required bandwidth, at least for the portion of the journey where the user does not request navigation information. Further, this reduces the unnecessary distraction of the user as unnecessary navigation information is not provided when the user is familiar with most of the route. For example, when a driver is lost or unsure about the next maneuver, the safety of the driver is improved by avoiding the need for the driver to start a navigation session in the middle of the journey. Moreover, in some scenarios, the ad-hoc navigation system further reduces the required bandwidth by caching the navigation guidance from a previous request and using the cached navigation guidance to respond to the request instead of obtaining the navigation guidance from the server.

[0010] An exemplary embodiment of the techniques of the present disclosure is a method for providing navigation information regarding a current journey of a user without the user having previously initiated a navigation session. The method includes receiving a navigation information request regarding the current journey of the user prior to the user initiating a navigation session, and identifying a destination of the current journey. The method also includes generating one or more sets of navigation guidance for moving along one or more routes from the current position of the user to the destination based on the current trajectory of the user, and providing a response to the navigation information request based on the generated one or more sets of navigation guidance.

[0011] Another exemplary embodiment is a computing device for providing navigation information regarding a user's current itinerary when the user has not previously initiated a navigation session. The computing device includes one or more processors and a computer-readable memory, optionally non-transitory, that is connected to the one or more processors and stores instructions that, when executed by the one or more processors, cause the computing device to receive a navigation information request regarding the user's current itinerary before the user initiates a navigation session and to identify a destination of the current itinerary. Also, the instructions cause the computing device to generate one or more sets of navigation guidance for moving along one or more routes from the user's current location to the destination based on the user's current trajectory and to provide a response to the navigation information request based on the generated one or more sets of navigation guidance.

[0012] Yet another exemplary embodiment is a computer-readable medium, optionally non-transitory, that stores instructions for providing navigation information regarding a user's current itinerary when the user has not previously initiated a navigation session and that, when executed by one or more processors, cause the one or more processors to receive a navigation information request regarding the user's current itinerary before the user initiates a navigation session and to identify a destination of the current itinerary. Also, the instructions cause the one or more processors to generate one or more sets of navigation guidance for moving along one or more routes from the user's current location to the destination based on the user's current trajectory and to provide a response to the navigation information request based on the generated one or more sets of navigation guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

[0014] Overview Generally, techniques for providing navigation information regarding a user's current itinerary to a user in a state where the user has not previously started a navigation session can be implemented in a system including one or more user computing devices, one or more network servers, or a combination of these devices. However, for clarity, the following examples mainly focus on embodiments in which a user moving in the current itinerary provides a navigation information request to a user computing device. The user computing device analyzes the request using, for example, natural language processing techniques and determines whether a destination is included in the request. The user computing device can also analyze user context data such as previous requests and / or calendar events, the user's usual destination at a specific day / time to identify the destination of the current itinerary.

[0015] The user computing device can also identify the current location and / or current trajectory of the user computing device using, for example, sensor data from sensors within the user computing device. For example, the current trajectory can indicate that the user is heading east on Highway 101. The user computing device can provide the current location, current trajectory, and / or destination to an external server.

[0016] The external server may generate a navigation guidance set (s) from the current location to the destination, and the navigation guidance set (s) includes the current trajectory. In some embodiments, the external server then analyzes the navigation guidance set (s) using the attributes of the request and generates a response to the request. Next, the external server may provide the response to the request to the user computing device. In other embodiments, the external server provides the generated navigation guidance set (s) to the user computing device. Next, the user computing device analyzes the navigation guidance set (s) using the attributes of the request to generate a response to the request. Next, the user computing device presents the response to the user via the user interface of the user computing device in an audio response and / or a visual response.

[0017] Exemplary Hardware Components and Software Components Referring to FIG. 1, an exemplary communication system 100 in which techniques for providing ad hoc navigation information may be implemented includes a user computing device 102. The user computing device 102 may be a portable device such as, for example, a smartphone or a tablet computer. Also, the user computing device 102 may be a laptop computer, a desktop computer, a personal digital assistant (PDA), a wearable device such as a smartwatch or smart glasses, a virtual reality headset, etc. In some embodiments, the user computing device 102 may be removably attached to a vehicle, may be incorporated into a vehicle, and / or may be interactable with a head unit of the vehicle to provide navigation instructions.

[0018] The user computing device 102 may include one or more processors 104 and a memory 106 storing machine-readable instructions executable by the processor(s) 104. The processor(s) 104 may include one or more general-purpose processors (e.g., CPUs) and / or dedicated processing units (e.g., graphics processing units (GPUs)). The memory 106 may optionally be non-transitory memory and may include one or more suitable memory modules such as random access memory (RAM), read-only memory (ROM), flash memory, or other types of persistent memory. The memory 106 may store instructions for implementing a navigation application 108, and the navigation application 108 may provide navigation guidance (e.g., by displaying guidance via the user computing device 102 or issuing voice instructions), display an interactive digital map, request and receive routing data to provide driving guidance, walking guidance, or other navigation guidance, and provide various geospatial content such as traffic, points of interest (POIs), and weather information. The embodiments described herein for providing ad hoc navigation information include driving information, but the techniques can be applied to navigation information for any suitable means of transportation such as walking, cycling, or public transportation.

[0019] The navigation application 108 may include an ad hoc navigation response engine 160. When the user has not started a navigation session regarding the current trip, the ad hoc navigation response engine 160 may receive a voice or text request for navigation information regarding the user's current trip from the user. The user may provide a specific trigger phrase or hot word such as "Hey, Map" to cause the ad hoc navigation response engine 160 to receive a voice request from the user. Next, the ad hoc navigation response engine 160 may analyze the request using natural language processing techniques described below and identify the destination and / or other attributes included in the request.

[0020] If the request does not include a destination, the ad - hoc navigation response engine 160 may obtain a navigation request from a previous user and may infer the destination according to the destination within the previous navigation request. For example, the ad - hoc navigation response engine 160 may obtain a previous navigation request stored in the user computing device 102 or the external server 120. Also, the ad - hoc navigation response engine 160 may obtain user context data stored in the user computing device 102 or the external server 120, such as calendar data, normal destinations at a specific day / time, etc., and infer the destination.

[0021] In response to identifying or inferring the destination, the ad - hoc navigation response engine 160 may send a request for navigation guidance along the current trajectory from the user's current location to the destination to the external server 120. Next, the ad - hoc navigation response engine 160 may receive one or more navigation guidance sets (if multiple) from the external server 120 and may analyze the navigation guidance set(s) to provide a response to the navigation information request. In other embodiments, the ad - hoc navigation response engine 160 may obtain, for example, offline navigation guidance set(s) cached in the user computing device 102 and obtained from the external server 120 in a previous request.

[0022] Furthermore, memory 102 may include a language processing module 109a configured to implement and / or support the techniques of the present disclosure for providing ad-hoc navigation information via natural conversation. That is, language processing module 109a may include an automatic speech recognition (ASR) engine 109a1 configured to transcribe speech input from a user into a text set. Further, language processing module 109a may include a text-to-speech (TTS) engine 109a2 configured to convert the text into an audio output such as an audio response, an audio request, a navigation instruction, and / or other output for the user. In some scenarios, language processing module 109a may include a natural language processing (NLP) model 109a3 configured to output a text transcription, intent interpretation, and / or audio output related to speech input received from a user of user computing device 102. As described herein, for the purposes of transcribing user speech input into a text set, converting text output into an audio output, and / or performing any other suitable functions described herein as part of a conversation between user computing device 102 and the user, it should be understood that ASR engine 109a1 and / or TTS engine 109a2 may be included as part of NLP model 109a3.

[0023] Typically, the language processing module 109a may include computer-executable instructions for training and operating the NLP model 109a3. Typically, the language processing module 109a may establish a network architecture or topology and add layers associated with one or more activation functions (e.g., rectified linear units, softmax, etc.), loss functions, and / or optimization functions to train one or more NLP models 109a3. Such training may typically be performed using notation, machine learning (ML) models, and / or any other suitable training method. More generally, the language processing module 109a may train the NLP model 109a3 to perform two techniques, syntactic analysis and semantic analysis, that enable words spoken by the user and / or words generated by a text-to-speech conversion program (e.g., TTS engine 109a2) executed by the processor 104 to be understood by the user computing device 102 and / or any other suitable device (e.g., vehicle computing device 150).

[0024] Syntactic analysis typically involves using basic grammar rules to analyze text and identify the overall sentence structure, how specific words within the sentence are organized, and the relationships between words within the sentence. Syntactic analysis may include one or more subtasks such as tokenization, part-of-speech (PoS) tagging, parsing, lemmatization and stemming, stop word removal, and / or any other suitable subtask, or combinations thereof. For example, the NLP model 109a3 may use syntactic analysis to generate a text transcription from the user's speech input. Additionally or alternatively, the NLP model 109a3 may receive such a text transcription as a text set from the ASR engine 109a1 to perform semantic analysis on the text set.

[0025] Semantic analysis typically involves analyzing text in order to understand the meaning of the text and / or incorporate it in another manner. Specifically, the NLP model 109a3 that applies semantic analysis can study the meaning of individual words included in a text transcription in a process known as lexical semantics. Using these individual meanings, the NLP model 109a3 can then examine various combinations of words included in the sentences of the text transcription to identify one or more contextual meanings of the words. Semantic analysis can include one or more subtasks, such as semantic ambiguity resolution, relation extraction, sentiment analysis, and / or any other suitable subtask, or a combination thereof. For example, the NLP model 109a3 can use semantic analysis to generate one or more interpretations of intent based on a text transcription from syntactic analysis.

[0026] In these aspects, the language processing module 109a can include an artificial intelligence (AI)-trained conversation algorithm (e.g., a natural language processing (NLP) model 109a3) configured to interact with a user accessing the navigation app 108. The user can be directly connected to the navigation app 108 to provide verbal input / verbal responses (e.g., speech input), and / or the user requests can include text input / text responses, and the TTS engine 109a2 (and / or other suitable engine / model / algorithm) can convert the text input / text responses to voice input / voice responses for interpretation by the NLP model 109a3. When the user accesses the navigation app 108, the input / responses spoken by the user and / or the input / responses generated by the TTS engine 109a2 (or other suitable algorithm) are analyzed by the NLP model 109a3, and a text transcription and an interpretation of intent can be generated.

[0027] The language processing module 109a may train one or more NLP models 109a3 using multiple training speech inputs from multiple users to apply these NLP techniques and / or other NLP techniques. As a result, the NLP model 109a3 may be configured to output a text transcription and an intent interpretation corresponding to the text transcription based on syntactic and semantic analysis of the user's speech input.

[0028] In certain aspects, one or more types of machine learning (ML) may be used by the language processing module 109a to train the NLP model(s) 109a3. ML may be used by an ML module 109b that may store an ML model 109b1. The ML model 109b1 may be configured to receive a text set corresponding to a user input and output an intent based on the text set. The NLP model(s) 109a3 may be, and / or may include, one or more types of ML models such as the ML model 109b1. More specifically, in these aspects, the NLP model 109a3 may be a machine learning model (e.g., a large language model (LLM)) trained by the ML module 109b using one or more training data sets of text to output one or more training intents and one or more training destinations, as further described herein, or may include these. For example, an artificial neural network, a recurrent neural network, a deep learning neural network, a Bayesian model, and / or any other suitable ML model 109b1 may be used for training and / or otherwise implementing the NLP model(s) 109a3. In these aspects, training may be performed by iteratively training the NLP model(s) 109a3 using labeled training samples (e.g., training user inputs).

[0029] In the case where the NLP model(s) 109a3 is an artificial neural network, training of the NLP model(s) 109a3 can generate weights of by-products or parameters that can be initialized with random values. When the network is repeatedly trained using one of several gradient descent algorithms, the weights are modified, the loss is reduced, and the values output by the network can converge to expected values, i.e., "learned" values. In an embodiment, a regression neural network lacking an activation function can be selected, the input data can be normalized by mean centering, the loss can be identified, and the accuracy of the output can be quantified. Such normalization can use a mean squared error loss function and mean absolute error. The artificial neural network model can be verified and cross-validated using standard techniques such as holdout, K-fold, etc. In an embodiment, multiple artificial neural networks can be trained separately, operate separately, and / or be trained separately and operate in conjunction with each other.

[0030] In an embodiment, one or more NLP models 109a3 may include an artificial neural network having an input layer, one or more hidden layers, and an output layer. Each of the layers of the artificial neural network may include any number of neurons. The multiple layers may be linearly connected to each other neuron by neuron, passing the output from one neuron to the next, or may be networked together such that the neurons transmit inputs and outputs non-linearly. It should be understood that generally many configurations and / or connections of the artificial neural network are possible. For example, the input layer may correspond to input parameters given as a complete sentence, or input parameters separated according to word or character (e.g., fixed-width) constraints. The input layer may, in some embodiments, correspond to a large number of input parameters (e.g., one million inputs) and may be analyzed sequentially or in parallel. Further, the various neurons and / or neuron connections within the artificial neural network may be initialized with any number of weights and / or other training parameters. Each of the neurons in the hidden layer may analyze one or more of the input parameters from the input layer and / or one or more of the outputs from one or more preceding layers of the hidden layer to generate a determination or other output. The output layer may include one or more outputs, each indicating a prediction. In some embodiments and / or scenarios, the output layer includes only a single output.

[0031] FIG. 1 shows the navigation application 108 as a stand-alone application, but note that the functionality of the navigation application 108 can also be provided in the form of an online service accessible via a web browser executed on the user computing device 102, as a plug-in or extension to other software applications executed on the user computing device 102, etc. The navigation application 108 can typically be provided in different versions for different operating systems. For example, a manufacturer of the user computing device 102 can provide a software development kit (SDK) that includes the navigation application 108 for the Android (trademark) platform, another SDK for the iOS (trademark) platform, etc.

[0032] Memory 106 may also store an operating system (OS) 110, which may be any type of suitable mobile or general-purpose operating system. User computing device 102 may further include one or more sensors such as a global positioning system (GPS) 112 or other suitable positioning module, an accelerometer, a gyroscope, a compass, an inertial measurement unit (IMU), a network module 114, a user interface 116 for displaying map data and guidance, and an input / output (I / O) module 118. Network module 114 may include one or more communication interfaces, such as hardware, software, and / or firmware of an interface that enables communication via any other suitable network such as a cellular network, a Wi-Fi network, or network 144 described below. I / O module 118 may include I / O devices capable of receiving input from the surrounding environment and / or the user and providing output to the surrounding environment and / or the user. I / O module 118 may include a touch screen, a display, a keyboard, a mouse, buttons, keys, a microphone, a speaker, etc. In various embodiments, user computing device 102 may include fewer components than illustrated in FIG. 1, or conversely, may include even more components.

[0033] The user computing device 102 can communicate with the external server 120 and / or the vehicle computing device 150 via the network 144. The network 144 can include one or more of an Ethernet-based network, a private network, a cellular network, a local area network (LAN), and / or a wide area network (WAN) such as the Internet. The navigation application 108 can send map data, navigation guidance, and other location-specific content to the vehicle computing device 150 for display on the cluster display unit 151. Additionally, the user computing device 102 may be directly connected to the vehicle computing device 150 via any suitable direct communication link 140, such as a wired connection (e.g., a USB connection).

[0034] In certain aspects, the network 144 can include any communication link suitable for short-range communication and can comply with communication protocols such as Bluetooth (trademark) (e.g., BLE), Wi-Fi (e.g., Wi-Fi Direct), NFC, ultrasonic signals, etc. Additionally or alternatively, the network 144 can be, for example, Wi-Fi, a cellular communication link (e.g., compliant with 3G, 4G, or 5G standards). In some scenarios, the network 144 can also include a wired connection.

[0035] External server 120 can be a remotely located server that includes the processing capabilities and executable instructions necessary to perform some or all of the actions described herein with respect to user computing device 102. For example, external server 120 can include a language processing module 120a similar to language processing module 109a included as part of user computing device 102, and module 120a can include one or more of ASR engine 109a1, TTS engine 109a2, and / or NLP model 109a3. External server 120 can also include a navigation app 120b and an ML module 120c similar to navigation app 108 and ML module 109b included as part of user computing device 102.

[0036] Ad-hoc navigation response engine 160 and navigation app 120b can operate as components of an ad-hoc navigation system. Alternatively, the ad-hoc navigation system can include only server-side components and simply provide a response to a navigation information request to ad-hoc navigation response engine 160. In other words, the ad-hoc navigation techniques in these embodiments can be implemented transparently to ad-hoc navigation response engine 160. As another alternative, all of the functionality of navigation app 120b can be implemented in ad-hoc navigation response engine 160.

[0037] Similarly, the language processing modules 109a, 120a may include separate components in the user computing device 102 and the external server 120, may include only server-side components, and provide language processing output to the ad-hoc navigation response engine 160, or all functions of the language processing modules 109a, 120a may be implemented in the user computing device 102. Further, the ML modules 109b, 120c may include separate components in the user computing device 102 and the external server 120, may include only server-side components, and provide ML output to the language processing modules 109a, 120a, or all functions of the ML modules 109b, 120c may be implemented in the user computing device 102.

[0038] The vehicle computing device 150 includes one or more processors 152 and a memory 153 that stores computer-readable instructions executable by the processor(s) 152. The memory 153 may store language processing module 153a, navigation application 153b, and ML module 153c, which are similar to the language processing module 109a, navigation application 108, and ML module 109b, respectively. The navigation application 153b may support functions similar to the vehicle-side navigation application 108 and may facilitate the rendering of information displays as described herein. For example, in certain aspects, the user computing device 102 may provide the acceptance route accepted by the user and the corresponding navigation instructions to be provided to the user as part of the acceptance route to the vehicle computing device 150. Next, the navigation application 153b may proceed to render the navigation instructions within the cluster unit display 151 and / or generate an audio output that provides the navigation instructions to the user in spoken language via the language processing module 153a.

[0039] In any event, the external server 120 can be communicatively connected to various databases such as the map database 156, the traffic database 157, and the point of interest (POI) database 159, and from these databases, the external server 120 can obtain navigation-related data. The map database 156 can include map data such as map tiles, visual maps, road shape data, road type data, speed limit data, etc. The map database 156 can also include route data for providing navigation guidance such as driving guidance, walking guidance, cycling guidance, or public transportation transfer guidance. The traffic database 157 can store past traffic information as well as real-time traffic information. The POI database 159 can store descriptions, locations, images, and other information regarding landmarks or points of interest. FIG. 1 shows the databases 156, 157, and 159, but the external server 120 may be communicatively connected to more or, conversely, fewer databases. For example, the external server 120 can be communicatively connected to a database that stores weather data.

[0040] Exemplary Navigation Information Request FIG. 2 shows an exemplary scenario 200 in which a user requests navigation information without pre-starting a navigation session. The user request can be a voice request. More specifically, the user asks, "Hey, map, should I take the highway to get to my office?" (202). The user can be the driver in the vehicle 12, a front seat passenger, a rear seat passenger, etc. The exemplary scenarios in FIGS. 2-4 include requests related to driving guidance, but these are just a few examples for ease of illustration. The ad-hoc navigation system can be used for any suitable means of transportation including driving, walking, cycling, or public transportation.

[0041] Furthermore, while the exemplary scenarios described herein are generally in the context of a speech-based configuration, the ad-hoc navigation system can also be used in the context of a touch-based interface or a visual interface. For example, a user's navigation information request may be input via free-form text input or via UI elements (e.g., dropdown menus). The response to the user request may be presented to the user via the user interface. The embodiments disclosed herein described in the context of a speech-based interface may also be applicable in the context of a touch-based interface. All embodiments disclosed herein where the input or output is described in the context of a speech-based interface can be adapted for application in the context of a touch-based interface.

[0042] The exemplary vehicle 12 of FIG. 2 includes a client device 10 and a head unit 14. The client device 10 communicates with the head unit 14 of the vehicle 12 via a communication link 16 that can be wired (e.g., Universal Serial Bus (USB)) or wireless (e.g., Bluetooth, Wi-Fi Direct). The client device 10 can also communicate with various content providers, servers, etc. via a wireless communication network such as a fourth-generation or third-generation cellular network (4G or 3G, respectively).

[0043] The head unit 14 may include a display 18 for presenting navigation information such as a digital map. In some embodiments, the display 18 is a touch screen and includes a software keyboard for entering text inputs that may include names or addresses such as a destination, a starting point, etc. The hardware input controls 20, 22 and the handle on the head unit 14 may be used to enter alphanumeric characters or to perform other functions for requesting navigation guidance. The head unit 14 may also include audio input components and audio output components such as a microphone 24 and a speaker 26, for example. The speaker 26 may be used to reproduce voice instructions or voice notifications transmitted from the client device 10.

[0044] In exemplary scenario 200, the user has not started a navigation session. In other words, before starting the current journey to the office, the user does not request navigation guidance from the user's current location to the user's office. In some embodiments, the user may not have launched the navigation application 108 and may simply start an interaction with the user computing device 102 by asking, "Hey, Maps, should I get on the highway to get to my office?" The phrase "Hey, Maps" can be a hotword or trigger to launch the navigation application 108. In other embodiments, the navigation application 108 may be launched and may continuously or periodically obtain the location data of the user computing device 102, for example, to identify the current trajectory of the user computing device 102, but may not be performing the execution of the navigation session and / or providing navigation guidance to the user's office. In other examples, "not having started a navigation session" may mean that the user computer device 102 is in a state that requires less battery usage and / or less processing power than when the navigation session is started. For example, the user computer device 102 may consume relatively more power when the navigation application 108 is launched compared to before launch.

[0045] In response to receiving the voice request for any event, the navigation application 108 may communicate with the language processing modules 109a, 120a in the user computing device 102 or the external server 120 to interpret the voice request.

[0046] More specifically, the user computing device 102 receives a voice request via an input device (e.g., a microphone that is part of the I / O module 118). Next, the user computing device 102 utilizes the processor 104 to execute instructions, included as part of the language processing module 109a, to transcribe the voice request into a text set. The user computing device 102 may cause the processor 104 to execute instructions, including, for example, an ASR engine (e.g., the ASR engine 109a1), to transcribe the voice request of the speech-based input received by the I / O module 118 to generate a text transcription of the user input. It should be understood that executing the ASR engine to transcribe the user input and generate a text transcription may be performed by the user computing device 102, the external server 120, the vehicle computing device 150, and / or any other suitable component, or a combination thereof.

[0047] Next, this transcription of the voice request may be analyzed by the processor 104 executing instructions, including, for example, the language processing module 109a and / or the machine learning module 109b, to interpret the text transcription and identify attributes of the request, such as the destination. The user computing device 102 may identify the destination by comparing terms within the voice request to POIs from the POI database 159, addresses included in the map database 156, or predefined destinations such as "home", "work", "my office" stored in the user profile.

[0048] In some embodiments, the voice request does not include a destination, or the language processing module 109a can identify a term corresponding to the destination, but this term does not specify a specific destination and refers to a destination category (e.g., "restaurant"). In these scenarios, the ad-hoc navigation response engine 160 can infer the destination using a previous navigation request that includes the destination and / or user context data, such as calendar data, normal destinations at a specific day / time, etc. For example, the ad-hoc navigation response engine 160 can obtain previous navigation requests and / or user context data stored in the user computing device 102 or the external server 120. Next, the ad-hoc navigation response engine 160 can analyze the previous navigation requests and / or user context data in consideration of the voice request to infer the destination.

[0049] For example, if the voice request includes the destination "restaurant" without specifying which restaurant, the ad-hoc navigation response engine 160 can analyze the most recent navigation request that includes a restaurant as the destination, the navigation guidance set(s), and / or the map data request. The ad-hoc navigation response engine 160 can also analyze the user context data to determine whether the user has a calendar event to go to a specific restaurant soon, or whether the user often goes to a specific restaurant based on the time / day (e.g., the user always goes to John's Pizza Restaurant for lunch on Tuesdays).

[0050] In some embodiments, based on recent navigation requests and context data, for example, when multiple restaurants may correspond to the “(requested (the)) restaurant”, the ad-hoc navigation response engine 160 may rank the restaurants that are candidate destinations. For example, candidate destinations may be ranked according to the currency of the request. More specifically, a navigation guidance request to a first restaurant 10 minutes before a voice request may result in a higher ranking of the first restaurant than a navigation guidance request to a second restaurant 1 hour before. Candidate destinations may be scored and / or ranked using any suitable factors (e.g., currency of the search, likelihood that the user will visit the candidate destination at a particular time / day, whether the user has a calendar event at the candidate destination, whether the candidate destination is along the user's current trajectory (e.g., is the user heading south and is the candidate destination south of the user's current location), etc.).

[0051] In some embodiments, the ad-hoc navigation response engine 160 may select the candidate destination with the highest rank or score as the destination and may presume that the candidate destination with the highest rank or score is the destination mentioned in the voice request. In other embodiments, the ad-hoc navigation response engine 160 may select a candidate destination only if the candidate destination has a score above a threshold score or a likelihood above a threshold likelihood of being the destination mentioned in the voice request. Alternatively, the ad-hoc navigation response engine 160 may provide a response to the user asking the user to clarify the destination.

[0052] In addition to identifying the destination in the user's navigation information request, the language processing module 109a can identify other attributes of the user request. For example, the language processing module 109a can identify the type of request, such as information regarding the next maneuver of the current itinerary, information regarding the travel time of the current itinerary, traffic information of the current itinerary, a navigation guidance set of the current itinerary, and the like. The language processing module 109a can also identify a specific maneuver within the request (e.g., a left turn, an entrance or exit of an arterial road), the location of the specific maneuver, a specific travel time for comparison with the remaining travel time or the travel time of the maneuver (e.g., "Arrive at the destination in 15 minutes?", "Still on the highway after 20 minutes?"), and specific parameters within the request.

[0053] In any event, the ad-hoc navigation response engine 160 can analyze the destination, attributes, and / or parameters included in or referred to by the user request in order to respond to the user request. More specifically, the ad-hoc navigation response engine 160 can obtain a navigation guidance set(s) from the user's current location to the specified or inferred destination (e.g., from an external server).

[0054] In some embodiments, the ad hoc navigation response engine 160 may identify the user's current trajectory based on the location data of the user computing device 102. For example, the ad hoc navigation response engine 160 may continuously or periodically obtain the current location of the user computing device 102 in response to a user request or when the navigation application 108 is launched. Next, the ad hoc navigation response engine 160 may identify that the user is moving in a particular direction on a particular road. Next, the ad hoc navigation response engine 160 may filter the navigation guidance sets from the user's current location to the destination to exclude navigation guidance sets that do not start by moving in the same direction and / or on the same road as the user. In this way, the ad hoc navigation response engine 160 may consider that the user's route to the destination includes the user's current trajectory.

[0055] Next, the ad hoc navigation response engine 160 may analyze the filtered navigation guidance set(s) to generate a response to the user's navigation information request. In exemplary scenario 200, the ad hoc navigation response engine 160 may identify that the destination of request 202 is the user's office. The ad hoc navigation response engine 160 may obtain the location of the user's office, for example, from a user profile stored in the user computing device 102 or the external server 120. Next, the ad hoc navigation response engine 160 may obtain the navigation guidance set(s) from the user's current location to the user's office. The ad hoc navigation response engine 160 may also identify that the user is currently heading east and may filter navigation guidance sets that do not start by heading east.

[0056] The ad-hoc navigation response engine 160 can also identify, via the language processing modules 109a, 120a, that the request type pertains to information regarding the next maneuver of the current itinerary, that the next maneuver pertains to a highway entrance, and that the highway entrance is the nearest highway and the nearest highway 90 to the destination. Next, the ad-hoc navigation response engine 160 can analyze the navigation guidance set(s) to determine whether any of the navigation guidance sets includes traveling on the highway 90. In some embodiments, the ad-hoc navigation response engine 160 can compare the navigation guidance set of the route that includes traveling on the highway 90 with an alternative route that does not include the highway 90. The ad-hoc navigation response engine 160 can identify attributes of the route, such as whether the route that includes the highway 90 is the fastest route, whether the route includes construction, or other attributes of the route.

[0057] Next, the ad-hoc navigation response engine 160 can generate a response to the user request, for example, based on the attributes of the route. For example, if the route including the arterial road 90 is the fastest route, the ad-hoc navigation response engine 160 can generate a response indicating that the user should pass through the highway. If there is an alternative route that can get the user to the user's office faster, the ad-hoc navigation response engine 160 can generate a response indicating that the user should avoid the highway. In other embodiments, the route including the arterial road 90 has a travel time similar to that of the alternative route (for example, the difference in travel time is 1 or 2 minutes, or the difference in travel time is less than the threshold difference), but if there is not much traffic or construction on the alternative route, the ad-hoc navigation response engine 160 can generate a response indicating that the user should avoid the highway. More generally, the response to the user request may include one or more sets of navigation guidance for moving to the destination, route information for moving to the destination, traffic information for moving to the destination, the travel time for the remaining part of the current journey to the destination, the travel time for a section of the route (for example, the arterial road section of the route), traffic information for a section of the route, etc. Prior to the generation of the response, the navigation guidance or movement information (such as the aforementioned information) may not be provided to the user. Thus, advantageously, the described process minimizes battery usage and optimizes processing efficiency by providing only navigation guidance or movement information as a response to the user request. Since the client device 10 or the head unit 14 may be required to provide a response to the user request as an output from such a device until such time, for example, they can turn off the power of their screen / display, enter a device sleep mode, or enter a device battery optimization mode, such advantages exist.

[0058] In exemplary scenario 200, the route including arterial road 90 is not the fastest route. Thus, the ad-hoc navigation response engine 160 generates a response 204 recommending that the user avoid the highway due to traffic. Next, the ad-hoc navigation response engine 160 may provide the response 204 to the user's navigation information request via audio output through a speaker, visual output on the user interface 116, and / or a combination of audio output / visual output.

[0059] The user computing device 102 may generate the text of the response 204 by utilizing the language processing module 109a and, in certain aspects, a large language model (LLM) (e.g., a language model for dialogue applications (LaMDA)) (not shown) included as part of the language processing module 109a. Such an LLM may be trained to generate response text based on the characteristics of the response and / or the LLM may be trained to receive the natural language expression of the response to the navigation information request as input and output a set of texts representing the audio response based on the characteristics.

[0060] In any event, once the user computing device 102 fully generates the text of the response 204, the device 102 may proceed to synthesize the text into speech for audio output to the user. Specifically, the user computing device 102 may send the text of the response 204 to a TTS engine (e.g., TTS engine 109a2) to provide audible output of the response 204 via a speaker (e.g., speaker 26). Additionally or alternatively, the user computing device 102 may also visually prompt the user so that the user can interact with the display screen (e.g., perform clicks, taps, swipes, etc.) and / or verbally acknowledge the response 204 by displaying the text of the response on a display screen (e.g., cluster display unit 151, user interface 116).

[0061] Figure 3 shows another exemplary scenario 300 where the user requests navigation information without first starting a navigation session. In this scenario 300, the user's request does not include a destination. More specifically, the user asks, "Hey, Map, which exit should I take from this roundabout?" (302). Similar to the exemplary scenario 200, the user has not first started a navigation session before providing this navigation information request.

[0062] In scenario 300, the user request does not include a destination. Thus, the ad - hoc navigation response engine 160 can infer the destination using previous navigation requests that include a destination and / or user context data, such as calendar data, normal destinations at a particular day / time, etc. For example, the ad - hoc navigation response engine 160 can identify that the user's most recent navigation request was for navigation guidance to John's Pizza Restaurant. In other embodiments, the ad - hoc navigation response engine 160 can identify that the user's calendar includes a meeting at John's Pizza Restaurant within a threshold time period from the current time (e.g., within 15 minutes, 30 minutes, 1 hour, etc.). As a result, the ad - hoc navigation response engine 160 can infer that the destination is John's Pizza Restaurant.

[0063] In addition to inferring the destination, the language processing module 109a can identify other attributes of the user request. For example, the language processing module 109a can identify the request type as information regarding the next maneuver of the current trip. The language processing module 109a can also identify specific parameters within the request, such as the roundabout exit for the next maneuver of the current trip.

[0064] Next, the ad-hoc navigation response engine 160 may obtain (e.g., from an external server) one or more sets of navigation guidance to a destination (John's Pizza Restaurant) inferred from the user's current location. The ad-hoc navigation response engine 160 may also identify the user's current trajectory based on the location data of the user computing device 102. For example, the ad-hoc navigation response engine 160 may continuously or periodically obtain the current location of the user computing device 102 in response to a user request when the navigation application 108 is launched, or in any other appropriate situation, e.g., when the user computing device 102 detects that the user is moving (e.g., the speed of the user computing device 102 exceeds a threshold speed such as 10 mph). Next, the ad-hoc navigation response engine 160 may identify that the user is moving in a particular direction on a particular road.

[0065] Furthermore, the ad-hoc navigation response engine 160 may identify the user's current trajectory based on the user's navigation information request. More specifically, since the request indicates that the user is moving through a roundabout, the ad-hoc navigation response engine 160 may identify that the particular road on which the user is moving is a roundabout. The ad-hoc navigation response engine 160 may compare the trajectory obtained from the location data with the trajectory information (e.g., roundabout) indicated in the request to confirm that the trajectory obtained from the location data matches the trajectory information indicated in the request. If there is a mismatch, the ad-hoc navigation response engine 160 may adjust the trajectory obtained from the location data to a nearby road that matches the trajectory information indicated in the request, or may provide the user with a response requesting clarification regarding the request.

[0066] For any event, next, the ad-hoc navigation response engine 160 may filter the navigation guidance set from the user's current location to the destination to exclude navigation guidance sets that do not include roundabouts and / or do not include exiting a roundabout in the next maneuver. Next, the ad-hoc navigation response engine 160 may analyze the filtered navigation guidance set(s) to generate a response to the user's navigation information request. More specifically, the ad-hoc navigation response engine 160 may analyze the filtered navigation guidance set(s) each including exiting a roundabout to identify the roundabout exit for moving to John's Pizza Restaurant. In some scenarios, the ad-hoc navigation response engine 160 may identify the same roundabout exit (e.g., the third roundabout exit) in each of the filtered navigation guidance set(s), and may identify this as the exit for going to John's Pizza Restaurant.

[0067] In other scenarios, the ad-hoc navigation response engine 160 may identify different roundabout exits in different navigation guidance sets. In these scenarios, the ad-hoc navigation response engine 160 may identify roundabout exits with respect to the fastest route to John's Pizza Restaurant, the shortest route to John's Pizza Restaurant, a route to avoid traffic or construction, etc. Next, the ad-hoc navigation response engine 160 may provide the response 304 to the user's navigation information request as an audio output via a speaker, a visual output on the user interface 116, and / or a combination of audio output / visual output. The response 304 may indicate the destination (e.g., to reach the restaurant, you need to exit from the third exit of the roundabout), whereby the user can confirm that the inferred destination is the correct destination. If the inferred destination is not the correct destination, the user may provide a follow-up request.

[0068] Figure 4 shows yet another exemplary scenario 400 in which the user requests navigation information without first starting a navigation session. Also in this scenario 400, the user request does not include a destination. More specifically, the user asks, "Hey, Map, how long will it take?" (402). As in exemplary scenarios 200, 300, the user has not first started a navigation session before providing this navigation information request.

[0069] In scenario 400, the user request does not include a destination. Accordingly, the ad-hoc navigation response engine 160 may infer a destination using a previous navigation request that includes a destination and / or user context data, such as calendar data, usual destinations at a particular day / time, etc. For example, the ad-hoc navigation response engine 160 may identify that the user usually returns home from work at the current time and / or day of the week (e.g., 5:00 p.m. on a Monday). As a result, the ad-hoc navigation response engine 160 may infer that the destination is the user's home.

[0070] In addition to inferring the destination, the language processing module 109a may identify other attributes of the user request. For example, the language processing module 109a may identify the request type as information regarding the travel time of the current trip. The language processing module 109a may also identify specific parameters within the request, such as the remaining time to reach the destination.

[0071] Next, the ad-hoc navigation response engine 160 may obtain one or more sets of navigation guidance to a destination (the user's home) inferred from the user's current location (e.g., from an external server). The ad-hoc navigation response engine 160 may also identify the user's current trajectory based on the location data of the user computing device 102. For example, the ad-hoc navigation response engine 160 may continuously or periodically obtain the current location of the user computing device 102 in response to a user request or when the navigation application 108 is launched. Next, the ad-hoc navigation response engine 160 may identify that the user is moving in a particular direction on a particular road.

[0072] Next, the ad-hoc navigation response engine 160 may filter the set(s) of navigation guidance from the user's current location to the destination to exclude sets of navigation guidance that do not start by moving in the same direction and on the same road as the user. Next, the ad-hoc navigation response engine 160 may analyze the filtered set(s) of navigation guidance to generate a response to the user's navigation information request. More specifically, the ad-hoc navigation response engine 160 may analyze the filtered set(s) of navigation guidance and / or the current traffic data for each route(s) to identify the travel time for each set of navigation guidance. Next, the ad-hoc navigation response engine 160 may select the travel time of the fastest route for inclusion in the response 404.

[0073] In other embodiments, the ad-hoc navigation response engine 160 may select the travel time of the most common route as the time required for inclusion in the response 404. More specifically, the fastest route may include some side streets and / or shortcuts that are not commonly recognized by most users. Other routes may include roads that are more likely to be traveled by users, such as arterial roads, and the ad-hoc navigation response engine 160 may select the travel time of this route as the most likely travel time for the user. In other examples, the most common route may be the route that the user normally travels according to the user's navigation history. The ad-hoc navigation response engine 160 may compare the filtered navigation guidance set(s) with the route(s) from the user's workplace to the user's home and identify among the filtered navigation guidance set(s) those that match the route that the user normally travels from workplace to home.

[0074] Next, the ad-hoc navigation response engine 160 may provide the response 404 to the user's navigation information request via voice output through a speaker, via visual output on the user interface 116, and / or in combination with voice output / visual output. The response 404 may include the selected travel time and may indicate that the user will arrive at the destination in about 30 minutes.

[0075] Figures 5-7 illustrate exemplary interactions between the user 502 and the user computing device 102 when the user requests navigation information prior to starting a navigation session. In the exemplary scenario 500 of Figure 5, the user 502 does not include a destination in the request. More specifically, the user 502 asks "Hey, Map, should I get off the bus at the next stop?" (504a).

[0076] Next, the ad-hoc navigation response engine 160 can infer the destination using previous navigation requests that include the destination and / or user context data, such as calendar data, usual destinations at specific days / times, etc. The ad-hoc navigation response engine 160 can identify multiple candidate destinations (e.g., the user's office and John's restaurant), and rank or score the candidate destinations using any suitable factors (e.g., recency of the search, likelihood of the user visiting the candidate destination at a specific time / day, whether the user has a calendar event at the candidate destination, etc.). In this scenario 500, none of the candidate destinations have a score exceeding a threshold score, a likelihood of being the destination mentioned in the request exceeding a threshold likelihood, and / or a score greater than the threshold but not exceeding the scores of other candidate destinations.

[0077] As a result, the user computing device 102 can send a first response 506 to the user 502 prompting the user 502 to clarify whether the destination is John's restaurant or the user's office (504b). Next, the user 502 can respond by sending a clarification response indicating that the destination is the user's office (504c). Next, the user computing device 102 can identify the user's current trajectory based on the location data of the user computing device 102 and / or the trajectory information included in the user's navigation information request (e.g., movement by bus). The user computing device 102 can also obtain a set(s) of navigation guidance from the user's current location to the user's office and can filter out and exclude sets of navigation guidance that do not include bus movement from the user's current location.

[0078] Next, user computing device 102 may analyze the filtered navigation guidance set(s) to generate a second response 508 to the user's navigation information request. In some scenarios, user computing device 102 may identify the same bus stop (e.g., Willoughby as the next stop) in each of the filtered navigation guidance set(s), and may identify this as the bus stop for the user to go to the office.

[0079] In other scenarios, user computing device 102 may identify different bus stops in different navigation guidance sets. In these scenarios, user computing device 102 may identify bus stops with respect to the fastest route to the user's office, the shortest route to the user's office, a route that avoids traffic or construction, etc. Next, user computing device 102 may provide the second response 508 to the user's navigation information request as an audio output via a speaker, a visual output on user interface 116, and / or a combination of audio output / visual output.

[0080] To give the user 502 enough time to consider an appropriate response without continuously listening to the interior of the vehicle, the user computing device 102 typically transmits a first response 506 including an audio request via the speaker 26 and then allows the user 502 several seconds (e.g., 5 - 10 seconds) to respond. By default, the user computing device 102 may disable the microphone and / or other listening devices (e.g., included as part of the I / O module 118) while the navigation app 108 is running and / or while processing information received via the microphone, e.g., by or according to the processor 104, the language processing module 109a, the machine learning module 109b, and / or the OS 110. Thus, except when the user computing device 102 provides an audio request to the user 502 and expects an oral response from the user 502 within a few seconds of transmission, the user computing device 102 may not actively listen to the vehicle interior during a navigation session and / or at any other time.

[0081] Figure 6 shows an exemplary scenario 600 similar to the exemplary scenario 500 of Figure 5. In the exemplary scenario 600, the user 502 asks, "Hey, Map, should I get off the bus at the next stop?" (604). This is the same user request as in the exemplary scenario 500. Accordingly, the user computing device 102 identifies a plurality of candidate destinations (e.g., the user's office and John's restaurant) in response to the user request, but none of the candidate destinations have a score exceeding a threshold score, a likelihood of being the destination mentioned in the request exceeding a threshold likelihood, and / or a score greater than the threshold that exceeds the scores of other candidate destinations.

[0082] However, instead of prompting user 502 to clarify whether the destination is John's restaurant or the user's office, user computing device 102 generates response 606 that provides navigation information for moving to both candidate destinations. More specifically, response 606 states, "To go to your office, get off the bus at Willoughby, the next stop. To go to John's restaurant, get off the bus at Main Street, three stops ahead." Next, user 502 can identify the stop based on user 502's destination. Instead of selecting one of the candidate destinations as the destination of response 606 or prompting user 502 to clarify the destination, user computing device 102 may include alternative responses for each candidate destination in response 606.

[0083] Figure 7 shows yet another exemplary scenario 700 where the user computing device prompts user 502 to start a navigation session in response to the user's navigation information request. In exemplary scenario 700, the user asks, "Hey, Map, how long will it take to get to Bill's house?" (704).

[0084] User computing device 102 can identify Bill's home address from Bill's contact information stored in user computing device 102, from previous navigation requests to Bill's home, from the pre-stored address of Bill's home, etc. Next, user computing device 102 can obtain a set (s) of navigation guidance from the user's current location to Bill's home.

[0085] The user computing device 102 can also identify the current trajectory of the user 502 based on the location data of the user computing device 102. For example, the user computing device 102 can identify that the user 502 is currently moving south on the arterial road 90. Next, the user computing device 102 can analyze the navigation guidance set(s) in consideration of the current trajectory of the user 502 to generate a response to the navigation information request of the user 502. In an exemplary scenario 700, the user computing device 102 can identify that it would take approximately 30 minutes to continue driving on the highway to Bill's house. However, the user computing device 102 can also identify that there is a faster route if the user 502 exits the arterial road 90 and moves on other roads to Bill's house.

[0086] Accordingly, the user computing device 102 can generate a response 706 to the request of the user 502 such that it indicates that the estimated time required to move to Bill's house on the highway is about 30 minutes. However, if the user 502 gets off the highway, there is a faster route 726. The response 706 can be presented as an audio response 706 and also as a text response 724 on the user interface 116 of the user computing device 102. Also, the text response 724 and / or the audio response 706 can include a prompt with user controls 724a, 724b for allowing the user 502 to select whether to receive turn-by-turn guidance for the faster route 726. In response to receiving a selection of the user control 724a indicating that the user 502 wants to start a navigation session (e.g., via a touch selection or voice input), the user computing device 102 can start a navigation session and provide navigation guidance for the faster route to Bill's house via voice guidance and / or via the navigation display 722 on the user computing device 102.

[0087] Exemplary Method for Providing Ad Hoc Navigation Information FIG. 8 is a flowchart of an exemplary method 800 for providing navigation information regarding a user's current itinerary in a state where the user has not previously initiated a navigation session, which may be implemented on a computing device such as the user computing device 102 of FIG. 1. Throughout the description of FIG. 8, the actions described as being performed by the user computing device 102 may, in some embodiments, be performed by the external server 120, the vehicle computing device 150, and / or may be performed in parallel by the user computing device 102, the external server 120, and / or the vehicle computing device 150. It should be understood that, for example, the user computing device 102, the external server 120, and / or the vehicle computing device 150 may utilize the language processing modules 109a, 120a, 153a, and / or the machine learning modules 109b, 120c, 153c to provide ad hoc navigation information.

[0088] Method 800 may be implemented as a set of instructions stored in a computer-readable memory and executable by one or more processors (e.g., processor(s) 104) of the user computing device 102. For example, method 800 may be executed by the ad hoc navigation response engine 160.

[0089] At block 802, the Ad-hoc Navigation Response Engine 160 may receive a navigation information request regarding the current itinerary before the user starts a navigation session. The request may be received as a voice request or via text input. The user may provide a specific trigger phrase or hotword such as "Hey, Map", whereby the Ad-hoc Navigation Response Engine 160 may receive a voice request from the user. In some embodiments, the user may be in a state where the navigation application 108 has not been launched prior to providing the request. In other embodiments, the navigation application 108 may be in a launched state and may continuously or periodically obtain the location data of the user computing device 102, for example, to identify the current trajectory of the user computing device 102, but may not be in a state of executing a navigation session and / or providing navigation guidance to a destination.

[0090] Next, at block 804, the Ad-hoc Navigation Response Engine 160 may identify the destination of the user's current itinerary. For example, the Ad-hoc Navigation Response Engine 160 may compare the terms in the navigation information request with POIs from the POI database 159, addresses included in the map database 156, or predetermined destinations such as "home", "work", "my office" stored in the user profile to identify the destination.

[0091] In some embodiments, the request does not include a destination, or the language processing module 109a may identify a term corresponding to the destination, but this term does not specify a specific destination and refers to a destination category (e.g., "restaurant"). In these scenarios, the Ad-hoc Navigation Response Engine 160 may infer the destination using previous navigation requests that include the destination and / or user context data, such as calendar data, normal destinations at a specific day / time, etc.

[0092] In some embodiments, based on recent navigation requests and context data, for example, when multiple destinations may correspond to the (requested) destination, the ad-hoc navigation response engine 160 may rank candidate destinations. The candidate destinations may be scored and / or ranked using any suitable factors (e.g., recency of the search, likelihood that the user will visit the candidate destination at a particular time / day, whether the user has a calendar event at the candidate destination, etc.).

[0093] In some embodiments, the ad-hoc navigation response engine 160 may select the candidate destination with the highest rank or score as the destination and may presume that the candidate destination with the highest rank or score is the destination mentioned in the voice request. In other embodiments, the ad-hoc navigation response engine 160 may select a candidate destination only if the candidate destination has a score exceeding a threshold score or a likelihood exceeding a threshold likelihood of being the destination mentioned in the voice request. Alternatively, the ad-hoc navigation response engine 160 may provide a response to the user asking the user to clarify the destination. In yet other embodiments, the ad-hoc navigation response engine 160 may select multiple destinations and may include in the response an alternative set of navigation information for each of the destinations.

[0094] At block 806, the ad-hoc navigation response engine 160 may generate a set of navigation guidance (s) for moving along a route (s) from the user's current location to a destination based on the user's current trajectory. For example, the ad-hoc navigation response engine 160 may communicate with the external server 120 to receive a set of navigation guidance (s) from the map database 156. In other embodiments, the ad-hoc navigation response engine 160 may obtain offline navigation guidance cached on the user computing device 102, for example, after receiving navigation guidance regarding a route to a destination. The ad-hoc navigation response engine 160 may identify the user's current trajectory based on the location data of the user computing device 102. For example, the ad-hoc navigation response engine 160 may continuously or periodically obtain the current location of the user computing device 102 in response to a user request or when the navigation application 108 is launched. Next, the ad-hoc navigation response engine 160 may identify that the user is moving in a particular direction along a particular road. The ad-hoc navigation response engine 160 may filter a set of navigation guidance from the user's current location to the destination to exclude a set of navigation guidance that does not start by moving in the same direction and on the same road as the user.

[0095] Next, at block 808, the ad-hoc navigation response engine 160 may provide a response to the navigation information request based on the generated navigation guidance set(s). The ad-hoc navigation response engine 160 may generate the text of the response by utilizing the language processing module 109a and, in certain aspects, a large language model (LLM) (e.g., LaMDA) included as part of the language processing module 109a. Such an LLM may be trained to generate response text based on the characteristics of the response and / or the LLM may be trained to receive, as input, the natural language expression of the response to the navigation information request and output a set of texts representing the voice response based on the characteristics.

[0096] In any event, once the ad-hoc navigation response engine 160 has fully generated the text of the response, the user computer device 102 may proceed to synthesize the text into speech for voice output to the user. Specifically, the user computing device 102 may send the text of the response to a TTS engine (e.g., TTS engine 109a2) for audible output of the response via a speaker (e.g., speaker 206) so that the user can listen to and interpret the response. Additionally or alternatively, the user computing device 102 may also visually prompt the user so that the user can interact with the display screen (e.g., perform clicks, taps, swipes, etc.) and / or verbally acknowledge receipt of the response by displaying the text of the response on a display screen (e.g., cluster display unit 151, user interface 116).

[0097] Other considerations The following additional considerations apply to the foregoing discussion. Throughout this specification, components, operations, or structures described as a single instance may be implemented by multiple instances. Individual operations of one or more methods are illustrated and described as separate operations, but one or more of the individual operations may be performed concurrently, and there is no need at all for the operations to be performed in the order illustrated. Structures and functions presented as separate components in an exemplary configuration may be implemented as a combined structure or component. Similarly, structures and functions presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements are included within the scope of the subject matter of this disclosure.

[0098] Furthermore, certain embodiments are described herein as including logic or some components, modules, or mechanisms. A module can be either a software module (e.g., code stored in a machine-readable medium) or a hardware module. A hardware module is a tangible unit capable of performing a particular operation and can be configured or arranged in a particular manner. In an exemplary embodiment, one or more computer systems (e.g., a stand-alone, client computer system, or server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) can be configured by software (e.g., an application or a portion of an application) as a hardware module that operates to perform the particular operations described herein.

[0099] In various embodiments, a hardware module can be implemented mechanically or electronically. For example, a hardware module can comprise dedicated circuitry or dedicated logic that is permanently configured (as a dedicated processor such as, for example, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)) to perform a particular operation. A hardware module can also comprise programmable logic or a programmable circuit that is temporarily configured by software to perform a particular operation (such as, for example, that is included within a general purpose processor or other programmable processor). It will be appreciated that the decision of whether to implement a hardware module mechanically within a permanently configured dedicated circuit or mechanically within a temporarily configured (e.g., software configured) circuit can be made considering cost and time.

[0100] Accordingly, the term hardware is to be understood to encompass a tangible entity, i.e., one that is physically constructed, permanently configured (e.g., physically incorporated), or temporarily configured (e.g., programmed) to operate in a particular manner as described herein or to perform a particular operation as described herein. As used herein, a “hardware-implemented module” refers to a hardware module. Considering embodiments in which a hardware module is temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one time instance. For example, if a hardware module includes a general purpose processor that is configured using software, the general purpose processor can be configured as respective different hardware modules at different times. Software can thus configure a processor, for example, to constitute a particular hardware module at one time instance and to constitute a different hardware module at a different time instance.

[0101] Hardware modules can provide information to other hardware and receive information from other hardware. Thus, the described hardware modules can be considered to be communicatively coupled. If there are multiple such hardware modules at the same time, communication can be achieved via signal transmissions that connect the hardware modules (e.g., via suitable circuits and buses). In embodiments where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, through the storage and retrieval of information in a memory structure accessible to the multiple hardware modules. For example, one hardware module can execute an operation and store the output of that operation in a communicatively connected memory device. Then, at a later time, another hardware module can access the memory device, retrieve the stored output, and process it. Hardware modules can also initiate communication with input or output devices and operate on resources (e.g., a collection of information).

[0102] Method 800 may include one or more functional blocks, modules, individual functions, or routines in the form of tangible computer-executable instructions, which are stored on a computer-readable storage medium, optionally a non-transitory computer-readable storage medium, and are executed using a processor of a computing device (e.g., a server device, a personal computer, a smartphone, a tablet computer, a smartwatch, a mobile computing device, or other client computing device as described herein). Method 800 may be included as part of any backend server (e.g., a map data server, a navigation server, or any other type of server computing device as described herein), for example, as part of a client computing device module in an exemplary environment, or as part of a module external to such an environment. The drawings may be described with reference to other drawings for ease of explanation, but Method 800 can be utilized with other objects and user interfaces. Further, in the above description, steps of Method 800 executed by a specific device (such as a user computing device) are described, but this is done for illustrative purposes only. Blocks of Method 800 may be executed by one or more devices or other parts of the environment.

[0103] Various operations of the exemplary methods described herein may be at least partially executed by one or more processors temporarily configured (e.g., by software) or permanently configured to perform the associated operations. Whether temporarily configured or permanently configured, such processors may form modules implemented with a processor that operates to perform one or more operations or functions. Modules referred to herein may, in some exemplary embodiments, include modules implemented with a processor.

[0104] Similarly, the methods or routines described herein can be implemented, at least in part, by a processor. For example, at least some of the operations of the method can be implemented by one or more processors or hardware modules in which the processor is implemented. The execution of certain operations of the operations may be distributed not only among the processors present within a single machine, but also among one or more processors arranged across multiple machines. In some exemplary embodiments, the processor(s) can be located in a single location (e.g., within a home environment, within an office environment, or as a server farm), but in other embodiments, the processors can be distributed across multiple locations.

[0105] One or more processors can also operate to support the execution of related operations in a "cloud computing" environment or as SaaS. For example, as shown above, at least some of the operations can be executed by a group of computers (as an example of a machine including a processor), and these operations are accessible via a network (e.g., the Internet) and via one or more suitable interfaces (e.g., an API).

[0106] Furthermore, the drawings illustrate some embodiments of exemplary environments for illustrative purposes only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein can be employed without departing from the principles described herein.

[0107] Upon reading this disclosure, through the principles disclosed herein, those skilled in the art will appreciate yet another alternative structural and functional design for identifying locations and routes through natural conversations. Thus, while specific embodiments and applications have been illustrated and described, it should be understood that the disclosed embodiments are not limited to the exact configurations and components disclosed herein. Various modifications, changes, and variations that will be apparent to those skilled in the art may be made to the arrangement, operation, and details of the methods and apparatuses disclosed herein without departing from the spirit and scope defined by the appended claims.

Claims

1. A method for providing navigation information regarding a current journey of a user in a state where the user has not previously started a navigation session, comprising: Receiving, by one or more processors, a navigation information request regarding a current journey of the user prior to the user starting a navigation session; Identifying, by the one or more processors, a destination of the current journey; Generating, by the one or more processors, one or more navigation guidance sets for moving along one or more routes from the current position of the user to the destination based on the current trajectory of the user; Providing, by the one or more processors, a response to the navigation information request based on the one or more generated navigation guidance sets. A method as described above.

2. The method according to claim 1, wherein the navigation information request is received prior to launching a navigation application.

3. The method according to claim 1 or 2, wherein the request does not include the destination of the current journey.

4. Identifying the destination of the current journey comprises: Identifying, by the one or more processors, the destination of the current journey as the destination included in the navigation information request; Inferring, by the one or more processors, the destination of the current journey based on a destination included in a previous request; or Inferring, by the one or more processors, the destination of the current journey based on user context data. The method according to any one of claims 1 to 3, comprising at least one of the above.

5. Providing a response to the navigation information request comprises: The one or more generated navigation guidance sets; Route information for moving to the destination; Traffic information for moving to the destination; or The time required for the remaining part of the current journey to the destination. The method according to any one of claims 1 to 4, comprising providing at least one of the above.

6. The destination is a destination included in a previous request, and generating the one or more navigation guidance sets comprises: obtaining, by the one or more processors, the one or more navigation guidance sets cached from the previous request The method according to any one of claims 1 to 5, comprising **Claim 7** The method according to any one of claims 1 to 6, wherein the request is a voice request and the response to the request is a voice response. **Claim 8** The method according to any one of claims 1 to 7, wherein the request relates to the time required for the remaining part of the current journey to a specific destination, and the response includes a notification of the time required for the remaining part of the current journey to the specific destination based on the most likely route to the specific destination according to the current trajectory and current traffic data on the route. **Claim 9** A computing device for providing navigation information regarding a current journey by a user without the user having previously initiated a navigation session, comprising: one or more processors; a computer-readable memory connected to the one or more processors and storing instructions; When the instructions are executed by the one or more processors, the instructions cause the computing device to: receive a navigation information request regarding the user's current journey before the user starts a navigation session; identify the destination of the current journey; generate one or more navigation guidance sets for moving along one or more routes from the user's current location to the destination based on the user's current trajectory; provide a response to the navigation information request based on the generated one or more navigation guidance sets; A computing device that performs **Claim 10** The computing device according to claim 9, wherein the navigation information request is received before launching a navigation application. **Claim 11** The computing device according to claim 9 or 10, wherein the request does not include the destination of the current journey. **Claim 12** To identify the destination of the current journey, the instructions cause the computing device to identifying the destination of the current trip as the destination included in the navigation information request; inferring the destination of the current trip based on the destination included in a previous request, or inferring the destination of the current trip based on user context data, The computing device according to any one of claims 9 to 11, wherein at least one of the above is executed.

13. To provide a response to the navigation information request, by the instructions, the computing device the generated one or more navigation guidance sets, route information for moving to the destination, traffic information for moving to the destination, or the time required for the remaining part of the current trip to the destination, The computing device according to any one of claims 9 to 12, wherein at least one of the above is provided.

14. The destination is the destination included in a previous request, and to generate the one or more navigation guidance sets, by the instructions, the computing device obtaining the one or more navigation guidance sets cached from the previous request, The computing device according to any one of claims 9 to 13, wherein the above is executed.

15. The computing device according to any one of claims 9 to 14, wherein the request is a voice request and the response to the request is a voice response.

16. The request relates to the time required for the remaining part of the current trip to a specific destination, and the response includes a notification of the time required for the remaining part of the current trip to the specific destination based on the most likely route to the specific destination according to the current trajectory and current traffic data on the route. The computing device according to any one of claims 9 to 15.

17. A computer-readable medium storing instructions for providing navigation information regarding a current trip by a user in a state where the user has not previously started a navigation session, wherein when the instructions are executed by one or more processors, the instructions cause the one or more processors to Receiving a navigation information request from the user regarding the current trip before the user starts a navigation session, identifying a destination of the current trip, generating one or more navigation guidance sets for moving along one or more routes from the current position of the user to the destination based on the current trajectory of the user, providing a response to the navigation information request based on the generated one or more navigation guidance sets, A computer-readable medium that performs the above.

18. The computer-readable medium according to claim 17, wherein the navigation information request is received prior to launching a navigation application.

19. The computer-readable medium according to claim 17 or 18, wherein the request does not include the destination of the current trip.

20. To identify the destination of the current trip, by the instructions, the one or more processors identifying the destination of the current trip as a destination included in the navigation information request, inferring the destination of the current trip based on a destination included in a previous request, or inferring the destination of the current trip based on user context data, The computer-readable medium according to any one of claims 17 to 19, which performs at least one of the above.

Images