Determining places and routes through natural conversations
Through natural dialogue interaction technology, users and navigation applications gradually refine destination and route selection, solving the problems of route selection difficulties and waste of computing resources in traditional navigation applications, and achieving a more efficient and safe navigation experience.
Patent Information
- Application Number
- CN202280095347.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-02
- Publication Date
- 2024-11-22
AI Technical Summary
When traditional navigation applications provide multiple route options, it is difficult for users to quickly find the right route, resulting in difficulty in selecting, waste of computing resources, and unsafe operation.
Through natural dialogue interaction, the dialogue interaction between users and navigation applications gradually refine destinations and route selections, provide a refined route list, and reduce user cognitive burden and computing resource needs.
It improves the efficiency and security of users' route selection, meets user preferences, provides a more efficient and secure navigation experience, and reduces computing resource consumption.
Smart Images

Figure CN119013535A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates generally to route determination, and more particularly to determining locations and routes through natural conversation. Background Art
[0002] The background description provided herein is for the purpose of presenting the context of the present disclosure in general. The work of the presently named inventors is neither explicitly nor implicitly admitted to be prior art to the present disclosure to the extent that it is described in this background section and in all aspects that may not have been described as prior art at the time of filing.
[0003] Generally speaking, traditional navigation applications that provide directions to and from a destination are ubiquitous in modern culture. These traditional navigation applications can provide directions and turn-by-turn navigation to reach a pre-programmed destination by driving and / or several other modes of transportation (e.g., walking, public transportation, etc.). Specifically, traditional navigation applications allow a user to specify a source point and a destination point, and then present the user with a set of route suggestions based on different modes of transportation. Therefore, these route suggestions are usually provided to the user as a result of a single interaction between the user and the application, in which the user enters a query and is subsequently presented with a list of route suggestions.
[0004] However, in situations where there are a variety of routes available or the user has flexibility in terms of exact destinations, this traditional single interaction method may not be sufficient. For example, a user may wish to navigate to a hiking trail without having to remember a specific hiking trail, and therefore there may be a large number of possible route configurations to reach multiple different hiking trails. If each of these route options were displayed, they could overwhelm the user or simply take too long to navigate, such that the user may select an undesired route or no route at all. Furthermore, displaying each of the large number of routes requires a correspondingly large amount of computing resources in order to determine and provide each of these routes at the client device.
[0005] Therefore, in general, traditional navigation applications are unable to provide users with route suggestions that are easy to understand and customized specifically for the users, and a navigation application that can determine locations and routes through natural conversations is needed to avoid these problems associated with traditional navigation applications. Summary of the invention
[0006] Using the technology of the present disclosure, the user's computing device can enable the user to select a destination and route through a back-and-forth dialogue with a navigation application. Specifically, the user can initiate a guidance query (e.g., a navigation session) through a spoken or typed natural language request, which may lead to follow-up questions from the navigation application and / or further refinement by the user. The navigation application using the present invention can then refine the route / destination provided to the user based on the user's answers to the follow-up questions. In this way, the present invention enables the user to refine his destination or route selection in a more natural way than traditional techniques through a two-way dialogue with his device. Therefore, the present invention can reduce the user's time and cognitive overhead because it does not need to browse a long list of different route suggestions and try to manually compare the route suggestions. In this way, the present invention solves the technical problem of efficiently determining the route to the destination. This is further benefited from the fact that the route provided to the user for selection is a refined list of all possible routes, which means that compared with traditional techniques, the computational requirements required to provide the user with a route are reduced because there are fewer routes to provide. In this way, the present invention provides a more computationally efficient method for determining the route to the destination. The additional technical advantage provided by the present invention is a safer way to provide users with routes for selection. Compared to conventional techniques of viewing routes displayed on a screen and selecting one of those routes through touch input, the disclosed techniques in which a user is able to refine a set of routes to a destination and select a route through voice input are less intrusive to the user. The disclosed techniques enable the operator of a vehicle to select a route without taking his eyes off the road or his hands off the vehicle controls. In addition, the user, as the operator of the vehicle, is able to safely refine or update the route using voice input and a conversational interface while traveling along the route. In this way, the disclosed techniques provide a safer method for selecting and refining a route to a destination. In addition, since users are encouraged to explicitly state their preferences in a conversational flow, the present invention can also provide route suggestions that better meet the needs and preferences of users than conventional techniques. However, embodiments of the present invention are not specifically limited to achieving effects based on user preferences. Some of the disclosures of the present invention are not related to user preferences.
[0007] The present invention can work in a setting based on voice or touch interface. However, for ease of discussion, the dialog flow between the user and the navigation application (and the corresponding processing component) described herein can generally be carried out in the context of a voice-based configuration. Nevertheless, in the case of a touch-based interface, clarification questions can be displayed to the user in the user interface, and the clarification questions can be answered through free-form text input or through UI elements (e.g., drop-down menus). The embodiments described in the context of a voice-based interface disclosed herein can also be applied to the context of a touch-based interface. All embodiments described in the context of a voice-based interface disclosed herein can be adapted to be applied to the context of a touch-based interface.
[0008] In a first example, the disclosed techniques can resolve locations in situations where the user has flexibility about the destination. The user may be traveling in Switzerland and may initiate a navigation session by saying "navigate to a nearby hiking spot." Given that there are a large number of hiking spots that satisfy the "nearby" constraint, the navigation system may respond to the user with an audio request: "What's the maximum amount of time you're willing to travel?" to narrow down the route. The user may respond to the audio request by stating "No more than 30 minutes by car." However, since there are still relatively many options available, the navigation application may generate a subsequent audio request, such as: "Some of the top rated options require taking a cable car from the parking lot, would you be willing to do that? The total journey time would likely be under 30 minutes. The user may respond with "Yes, that's fine", and the navigation application may respond with several options that are highly rated for hiking within a 30-minute driving time, but include both driving and a cable car. The user may then further refine the returned route options with subsequent statements, or accept one of the suggestions provided.
[0009] In a second example, the technology of the present disclosure may be configured to generate natural language route suggestions with refinement. A user may arrive at an Italian airport and want to navigate to their hotel. The user may ask the navigation application "Give me directions to Tenuta il Cigno." The navigation application may respond with several different route suggestions and the best candidate, stating: "The route I'd recommend is the shortestone but it involves driving 10 miles on a single track road. The user may not accept the suggestion, but may adjust the suggested route by saying "I'd definitely appreciate a short journey but is it possible to spend less time on the single track road? I certainly hope to travel a short distance, but is it possible to reduce the time spent on the single track? Then, the navigation application may suggest an alternative route that is longer but only involves driving 2 miles on a single track to reach the user's destination. The user may accept the alternative route and may view the guidance or start a navigation session.
[0010] In a third example, the technology of the present disclosure can provide dialogue clarification during a navigation session. Similar to the above example, a user may be navigating from an airport at a vacation destination to their hotel. While the user is en route, the user encounters a potential detour along the way that takes a similar amount of time but has different characteristics. When approaching the detour, the navigation system may prompt the user "There's an alternate route on the left with a similar ETA, it's a shorter distance but has some temporary road works which could cause a bit of a delay. (There is an alternate route on the left with a similar ETA, it's a shorter distance but has some temporary road works which could cause a bit of a delay.)" In response, the user may say "Ok let's take it" or "No, I think I'll stick to the current route", and the navigation application may continue the original route or switch to the alternate route as needed.
[0011] In this way, aspects of the present disclosure provide a technical solution to the problem of non-optimal route suggestions by automatically filtering route options based on a conversation between a user and a navigation application. Aspects of the present disclosure also provide technical effects for the problem of safer route refinement based on a conversational interaction between a user and a navigation application. Specifically, the conversational interaction requires less cognitive input from the user and is therefore less intrusive to the user because the user does not need to actually view and actually select a route on a display of the device. Instead, the user can verbally refine and select a route while driving or otherwise operating a vehicle. As previously described, conventional systems automatically provide a list of route options in response to a single query raised by a user. Therefore, conventional systems are severely limited in the search / determination criteria applied to generate a list of route options through a single query of the user, and are therefore greatly limited in the ability to refine the list of routes presented to the user. As a result, conventional systems provide a large number of possible routes, many of which are not optimized for the user's specific situation, thereby frustrating the user. In contrast, the technology of the present disclosure removes these frustrating, overwhelming interactions with navigation applications by talking to the user until the application has enough information to determine a refined set of optimal route suggestions, each of which is customized to the user's specific situation. Since the refined route set contains fewer routes than the original set, the refined best route suggestion set also requires less computing resources to process and provide to the user for selection. In this way, a more computationally efficient technique is disclosed compared to conventional techniques.
[0012] An example embodiment of the technology disclosed herein is a method in a computing device for determining a location and a route through a natural conversation. The method includes: receiving a voice input including a search query from a user to initiate a navigation session; generating a navigation search result set in response to the search query by one or more processors, the navigation search result set including multiple destinations or multiple routes corresponding to one or more destinations; providing an audio request to the user by one or more processors for refining the navigation search result set; receiving a subsequent voice input including a refined search query from the user in response to the audio request; and providing one or more refined navigation search results including a subset of the multiple destinations or the multiple routes in response to the refined search query by one or more processors.
[0013] Another example embodiment is a computing device for determining a place and a route through natural conversation. The computing device includes: a user interface; one or more processors; and a computer-readable memory, which is optionally non-transitory, coupled to the one or more processors and stores instructions thereon, which when executed by the one or more processors, causes the computing device to: receive a voice input including a search query from a user to initiate a navigation session; generate a navigation search result set in response to the search query, the navigation search result set including multiple destinations or multiple routes corresponding to one or more destinations; provide an audio request to the user for refining the navigation search result set; receive a subsequent voice input including a refined search query from the user in response to the audio request; and provide one or more refined navigation search results including a subset of the multiple destinations or the multiple routes in response to the refined search query.
[0014] Yet another example embodiment is a computer-readable medium, which is optionally non-transitory, storing instructions for determining locations and routes through natural conversation, which instructions, when executed by one or more processors, cause the one or more processors to: receive voice input including a search query from a user to initiate a navigation session; generate a navigation search result set in response to the search query, the navigation search result set including multiple destinations or multiple routes corresponding to one or more destinations; provide an audio request to the user for refining the navigation search result set; receive subsequent voice input including a refined search query from the user in response to the audio request; and provide one or more refined navigation search results including a subset of the multiple destinations or the multiple routes in response to the refined search query.
[0015] Another example embodiment is a method in a computing device for determining a location and a route through a natural conversation. The method includes: receiving input from a user to initiate a navigation session; generating one or more destinations or one or more routes in response to the user input; and providing a request to the user to refine the response to the user input. In response to the request, the method includes receiving subsequent input from the user, and providing one or more updated destinations or one or more updated routes in response to the subsequent user input. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1A is a block diagram of an example communication system in which techniques for determining location and route through natural conversation may be implemented;
[0017] Figure 1B An example vehicle interior is shown where a user can utilize Figure 1A a user computing device or a vehicle computing device to determine location and route through natural conversation;
[0018] Figure 2A Shows the user and Figure 1A example conversations between user computing devices to determine location and route through natural conversation;
[0019] Figure 2B A user input analysis sequence is shown to output an audio request and a navigation search result set;
[0020] Figure 2C A subsequent user input analysis sequence is shown to output a refined set of navigation search results;
[0021] Figure 3A An example transition between a user providing a route acceptance input and a user computing device displaying navigation instructions corresponding to the accepted route is shown;
[0022] Figure 3B An example route update sequence is shown to update navigation instructions provided to a user by prompting the user with an option to switch to an alternate route;
[0023] Figure 4 is a flow chart of an example method for determining location and route through natural conversation, which may be implemented in a computing device, such as the computing device of FIG. 1 . DETAILED DESCRIPTION
[0024] Overview
[0025] As previously described, navigation applications typically receive user input and automatically generate multiple route options from which the user can choose. However, in such situations, it is best to follow up with a clarifying question or statement with the user, thereby allowing the user to narrow down the set of possible routes to provide a reduced set of route choices. The technology of the present disclosure achieves this clarification by supporting a conversational route configuration that (i) detects situations where follow-up questions (referred to herein as "audio requests") would be beneficial, and (ii) provides the user with an opportunity to clarify their preferences in order to determine the best route. It will be appreciated that the technology of the present disclosure can also achieve clarification and best route suggestions in a manner that is independent of user preferences. For example, objectively safer, faster, or shorter route suggestions can be provided based on the conversational route configuration.
[0026] In general, a user's computing device may generate a refined set of navigation search results based on a series of inputs received from the user as part of a conversation with the user computing device. More specifically, the user computing device may receive a voice input including a search query from the user that initiates a navigation session. The navigation session generally corresponds to a set of navigation instructions that are intended to guide the user from a current location or a specified location to a destination, and such navigation instructions may be presented on a user interface for display to the user or audibly transmitted through an audio output component of the user computing device. The user computing device may then generate a set of navigation search results in response to the search query, and the set of navigation search results may include multiple destinations or multiple routes corresponding to one or more destinations.
[0027] At this point, the user computing device (e.g., via the navigation application) may determine that the navigation search result set can / should be refined before providing the search results to the user. For example, the user computing device may determine that the number of route options included in the navigation search result set is too large (e.g., exceeds a route presentation threshold) and may confuse and / or overwhelm the user, or that the computational cost of providing the search result set to the user is too high. Additionally or alternatively, the user computing device may determine that the best route included in the navigation instruction set has potential hazards and / or other unusual driving conditions that the user should be aware of before or during the navigation session. In any event, when the user computing device determines that the user should be prompted with an audio request, the user computing device may provide the user with an audio request to refine the navigation search result set.
[0028] Thus, in response to the audio request, the user computing device may receive a subsequent voice input from the user that includes a refined search query. This refined search query may include keywords or other phrases that may directly correspond to keywords or phrases included as part of the audio request, so that the user computing device may refine the navigation search result set based on the user's subsequent voice input. For example, the audio request provided by the user computing device to the user may prompt the user to specify the maximum driving time expected to reach the destination. In response, the user may say "I don't want to be on the road for more than 30 minutes. (I don't want to stay on the road for more than 30 minutes.)" The user computing device may receive such subsequent voice input from the user, interpret that 30 minutes is the expected maximum driving time, and filter the navigation search result set by deleting routes with an estimated driving time of more than 30 minutes. Thereafter, the user computing device may provide one or more refined navigation search results including a subset of multiple destinations or multiple routes in response to the refined search query.
[0029] In this way, aspects of the present disclosure provide a technical solution to the problem of non-optimal route suggestions by automatically filtering route options based on a conversation between a user and a navigation application. Conventional systems automatically provide a list of route options in response to a single query posed by a user and, therefore, are severely limited in the search / determination criteria applied to generate the list of route options and the ability to refine the list of routes provided to the user. Such conventional systems often frustrate users by providing a large number of possible routes, many of which are not optimized for the user's specific situation. In contrast, the technology of the present disclosure removes these frustrating, overwhelming interactions with the navigation application by conversing with the user until the application has enough information to determine a refined set of navigation search results, each of which is customized to the user's specific situation. The technology of the present disclosure provides a technical solution to the problem of optimizing computing resources when providing route suggestions by refining possible routes through a conversation with the user.
[0030] In addition, the present technology improves the overall user experience when utilizing navigation applications and, more generally, when receiving navigation instructions to a desired destination. The present technology automatically determines a refined set of navigation search results that, in some examples, are specifically customized / curated based on the user's preferences, such as determined through an intuitive and non-intrusive conversation between the user and their computing device. This helps provide a more user-friendly, relevant, and safe experience that increases user satisfaction with their travel plans, reduces user distractions when traveling to a destination, and reduces user confusion and frustration caused by non-optimized and / or irrelevant / inappropriate navigation suggestions of traditional navigation applications. Thus, the present technology enables a safer, more user-specific, and more enjoyable navigation session to a desired destination.
[0031] Example Hardware and Software Components
[0032] First reference Figure 1A , an example communication system 100 in which techniques for determining location and route through natural conversation may be implemented includes a user computing device 102. For example, the user computing device 102 may be a portable device, such as a smart phone or a tablet computer. The user computing device 102 may also be a laptop computer, a desktop computer, a personal digital assistant (PDA), a wearable device (such as a smart watch or smart glasses), etc. In some embodiments, the user computing device 102 may be removably mounted in a vehicle, embedded in a vehicle, and / or may be capable of interacting with a head unit of the vehicle to provide navigation instructions.
[0033] The user computing device 102 may include one or more processors 104 and a memory 106 storing machine-readable instructions executable on the processor 104. The processor 104 may include one or more general-purpose processors (e.g., CPUs) and / or special-purpose processing units (e.g., graphics processing units (GPUs)). The memory 106 may optionally be a non-transitory memory and may include one or more suitable memory modules, such as random access memory (RAM), read-only memory (ROM), flash memory, other types of persistent memory, etc. The memory 106 may store instructions for implementing a navigation application 108, which may provide navigation directions (e.g., by displaying directions or issuing audio instructions via the user computing device 102), display interactive digital maps, request and receive route data to provide driving, walking or other navigation directions, provide various geo-located content (such as traffic, points of interest (POIs) and weather information, etc.).
[0034] In addition, the memory 102 may include a language processing module 109a, which is configured to implement and / or support the technology of the present disclosure for determining places and routes through natural dialogue. That is, the language processing module 109a may include an automatic speech recognition (ASR) engine 109a1, which is configured to transcribe the voice input from the user into a text collection. In addition, the language processing module 109a may include a text-to-speech (TTS) engine 109a2, which is configured to convert text into audio output, such as audio requests, navigation instructions and / or other outputs of the user. In some scenarios, the language processing module 109a may include a natural language processing (NLP) model 109a3, which is configured to output text transcription, intent interpretation and / or audio output related to the voice input received from the user of the user computing device 102. It should be understood that, as described herein, the ASR engine 109a1 and / or the TTS engine 109a2 may be included as part of the NLP model 109a3 to transcribe user voice input into a text collection, convert text output into audio output, and / or any other suitable functionality described herein as part of a conversation between the user computing device 102 and the user.
[0035] In general, the language processing module 109a may include computer executable instructions for training and operating the NLP model 109a3. In general, the language processing module 109a may train one or more NLP models 109a3 by establishing a network architecture or topology and adding layers that may be associated with one or more activation functions (e.g., rectified linear units, softmax, etc.), loss functions, and / or optimization functions. Such training can typically be performed using symbolic methods, machine learning (ML) models, and / or any other suitable training methods. More generally, the language processing module 109a may train the NLP model 109a3 to perform two techniques: grammatical analysis and semantic analysis, which enable the user computing device 102 and / or any other suitable device (e.g., the vehicle computing device 151) to understand the words spoken by the user and / or the words generated by the text-to-speech program (e.g., TTS engine 109a2) executed by the processor 104.
[0036] Grammatical analysis generally involves analyzing text using basic grammatical rules to identify overall sentence structure, how specific words are organized within a sentence, and how words within a sentence relate to each other. Grammatical analysis may include one or more subtasks, such as tokenization, part-of-speech (PoS) tagging, parsing, lemmatization and stemming, stop word removal, and / or any other suitable subtasks or combinations thereof. For example, using grammatical analysis, the NLP model 109a3 may generate a text transcription based on the user's voice input. Additionally or alternatively, the NLP model 109a3 may receive such a text transcription as a text collection from the ASR engine 109a1 to perform semantic analysis on the text collection.
[0037] Semantic analysis generally involves analyzing text in order to understand and / or otherwise capture the meaning of the text. Specifically, the NLP model 109a3 to which semantic analysis is applied can study the meaning of each individual word contained in the text transcription in a process known as lexical semantics. Using these individual meanings, the NLP model 109a3 can then examine the various combinations of words included in the sentences of the text transcription to determine one or more contextual meanings of the words. Semantic analysis can include one or more subtasks, such as word sense disambiguation, relationship extraction, sentiment analysis, and / or any other suitable subtasks or combinations thereof. For example, using semantic analysis, the NLP model 109a3 can generate one or more intent interpretations based on the text transcription from the grammatical analysis.
[0038] In these aspects, the language processing module 109a may include an artificial intelligence (AI) trained dialog algorithm (e.g., a natural language processing (NLP) model 109a3) that is configured to interact with a user who is accessing the navigation app 108. The user may connect directly to the navigation app 108 to provide verbal input / responses (e.g., voice input), and / or the user request may include text input / responses, which the TTS engine 109a2 (and / or other suitable engines / models / algorithms) may convert into audio input / responses for interpretation by the NLP model 109a3. When the user accesses the navigation app 108, the input / responses spoken by the user and / or generated by the TTS engine 109a2 (or other suitable algorithms) may be analyzed by the NLP model 109a3 to generate text transcriptions and intent interpretations.
[0039] The language processing module 109a can train one or more NLP models 109a3 to apply these and / or other NLP techniques using multiple training speech inputs from multiple users. Thus, the NLP model 109a3 can be configured to output a text transcription and an intent interpretation corresponding to the text transcription based on grammatical analysis and semantic analysis of the user's speech input.
[0040] In certain aspects, the language processing module 109a may employ one or more types of machine learning (ML) to train the NLP model 109a3. ML may be employed by the ML module 109b, which may store the ML model 109b1. The ML model 109b1 may be configured to receive a text set corresponding to a user input and output an intent and a destination based on the text set. The NLP model 109a3 may be and / or include one or more types of ML models, such as the ML model 109b1. More specifically, in these aspects, the NLP model 109a3 may be or include a machine learning model (e.g., a large language model (LLM)) trained by the ML module 109b using one or more text training data sets to output one or more training intents and one or more training destinations, as further described herein. For example, an artificial neural network, a recurrent neural network, a deep learning neural network, a Bayesian model, and / or any other suitable ML model 109b1 may be used to train and / or otherwise implement the NLP model 109a3. In these aspects, training may be performed by iteratively training the NLP model 109a3 using labeled training samples (eg, training user input).
[0041] In the case where the NLP model 109a3 is an artificial neural network, the training of the NLP model 109a3 may produce byproduct weights, or parameters that may be initialized to random values. When the network is iteratively trained, the weights may be modified by using one of several gradient descent algorithms to reduce losses and converge the values of the network output to expected values or "learning" values. In an embodiment, a regression neural network lacking an activation function may be selected, wherein the input data may be normalized by mean centering to determine the loss and quantify the accuracy of the output. Such normalization may use a mean square error loss function and a mean absolute error. Standard techniques such as simple cross validation (hold-out), K-fold cross validation (K-fold) may be used to validate and cross-validate the artificial neural network model. In an embodiment, multiple artificial neural networks may be trained and operated individually, and / or multiple artificial neural networks may be trained and operated in combination individually.
[0042] In an embodiment, one or more NLP models 109a3 may include an artificial neural network with an input layer, one or more hidden layers, and an output layer. Each layer in the artificial neural network may include any number of neurons. Multiple layers may link neurons together linearly and may pass output from one neuron to the next neuron, or may be networked together so that neurons transmit input and output in a nonlinear manner. In general, it should be understood that many configurations and / or connections of artificial neural networks are possible. For example, the input layer may correspond to input parameters given as complete sentences, or input parameters separated according to word or character (e.g., fixed width) restrictions. In some embodiments, the input layer may correspond to a large number of input parameters (e.g., one million inputs), and may be analyzed serially or in parallel. In addition, various neurons and / or neuron connections within the artificial neural network may be initialized with any number of weights and / or other training parameters. One or more input parameters from the input layer may be analyzed in each neuron in the hidden layer, and / or one or more outputs from the previous one or more hidden layers to generate a decision or other output. The output layer may include one or more outputs, each indicating a prediction. In some embodiments and / or scenarios, the output layer includes only a single output.
[0043] It should be noted that although Figure 1AThe navigation application 108 is shown as a standalone application, but the functionality of the navigation application 108 may also be provided in the form of an online service accessible via a web browser executing on the user computing device 102, as a plug-in or extension of another software application executing on the user computing device 102, etc. The navigation application 108 may generally be provided in different versions for different operating systems. For example, the manufacturer of the user computing device 102 may provide a software development kit (SDK) for the Android™ platform (including the navigation application 108), another SDK for the iOS™ platform, etc.
[0044] The memory 106 may also store an operating system (OS) 110, which may be any type of suitable mobile or general purpose operating system. The user computing device 102 may also include a global positioning system (GPS) 112 or another suitable positioning module, a network module 114, a user interface 116 for displaying map data and directions, and an input / output (I / O) module 118. The network module 114 may include one or more communication interfaces, such as hardware, software, and / or firmware for interfaces for communicating via a cellular network, a Wi-Fi network, or any other suitable network (such as the network 144 discussed below). The I / O module 118 may include an I / O device capable of receiving input from the surrounding environment and / or a user and presenting output to the surrounding environment and / or a user. The I / O module 118 may include a touch screen, a display, a keyboard, a mouse, buttons, keys, a microphone, a speaker, and the like. In various implementations, the user computing device 102 may include, for example, a user interface 116 for displaying map data and directions. Figure 1A Fewer components may be shown, or conversely, additional components may be included.
[0045] The user computing device 102 may communicate with the external server 120 and / or the vehicle computing device 150 via a network 144. The network 144 may include one or more of an Ethernet-based network, a private network, a cellular network, a local area network (LAN), and / or a wide area network (WAN) (such as the Internet). The navigation application 108 may transmit map data, navigation directions, and other geo-positioned content from a map database 156 to the vehicle computing device 150 for display on a dashboard display unit 151. Additionally or alternatively, the navigation application 108 may access maps, navigation, and geo-positioned content stored locally at the user computing device 102, and may periodically access the map database 156 to update local data or access real-time information, such as real-time traffic data, during navigation. Furthermore, the user computing device 102 may be directly connected to the vehicle computing device 150 via any suitable direct communication link 140, such as a wired connection (e.g., a USB connection).
[0046] In certain aspects, the network 144 may include any communication link suitable for short-range communication and may conform to a communication protocol such as, for example, Bluetooth™ (e.g., BLE), Wi-Fi (e.g., Wi-Fi Direct), NFC, ultrasonic signals, etc. Additionally or alternatively, the network 144 may be, for example, Wi-Fi, a cellular communication link (e.g., conforming to 3G, 4G, or 5G standards), etc. In some scenarios, the network 144 may also include a wired connection.
[0047] The external server 120 may be a remotely located server that includes processing capabilities and executable instructions required to perform some / all of the actions described herein with respect to the user computing device 102. For example, the external server 120 may include a language processing module 120a similar to the language processing module 109a included as part of the user computing device 102, and the module 120a may include one or more of an ASR engine 109a1, a TTS engine 109a2, and / or an NLP model 109a3. The external server 120 may also include a navigation app 120b and an ML module 120c, which are similar to the navigation app 108 and the ML module 109b included as part of the user computing device 102.
[0048] The vehicle computing device 150 includes one or more processors 152 and a memory 153 storing computer-readable instructions executable by the processor 152. The memory 153 can store a language processing module 153a, a navigation application 153b, and an ML module 153c, which are similar to the language processing module 153a, the navigation application 108, and the ML module 109b, respectively. The navigation application 153b can support functions similar to the navigation application 108 from the vehicle side, and can facilitate the rendering of information displays, as described herein. For example, in some aspects, the user computing device 102 can provide the vehicle computing device 150 with an accepted route that has been accepted by the user, and corresponding navigation instructions provided to the user as part of the accepted route. The navigation application 153b can then continue to present the navigation instructions within the dashboard unit display 151 and / or generate an audio output, which verbally provides the navigation instructions to the user via the language processing module 153a.
[0049] Regardless, the user computing device 102 may be communicatively coupled to various databases from which the user computing device 102 may retrieve navigation-related data, such as a map database 156, a traffic database 157, and a point of interest (POI) database 159. The map database 156 may include map data, such as map tiles, visual maps, road geometry data, road type data, speed limit data, etc. The traffic database 157 may store historical traffic information as well as real-time traffic information. The POI database 159 may store descriptions, locations, images, and other information about landmarks or points of interest. Although Figure 1A Databases 156, 157, and 159 are depicted, but the user computing device 102, the vehicle computing device 150, and / or the external server 120 may be communicatively coupled to additional databases, or conversely, may be communicatively coupled to fewer databases. For example, the user computing device 102 and / or the vehicle computing device 150 may be communicatively coupled to a database storing weather data.
[0050] Go to Figure 1B , the user computing device 102 may transmit information to present / display navigation instructions within the vehicle environment 170. The user computing device 102 may be located within the vehicle 172 and may be a smartphone. However, although Figure 1B User computing device 102 is depicted as a smartphone, but this is for ease of illustration only and user computing device 102 may be any suitable type of device and may include any suitable type of portable or non-portable computing device.
[0051] Regardless, the vehicle 172 may include a head unit 174, which in some aspects may include and / or otherwise house a user computing device 102. Even if the head unit 174 does not include a user computing device 102, the device 102 may communicate with the head unit 174 (e.g., via a wireless or wired connection) to transmit navigation information (such as a map or audio instructions and / or information display) to the head unit 174 for display or transmission by the head unit 174. Additionally, the vehicle 172 includes a dashboard display unit 151, which may display information transmitted from the user computing device 102. In some aspects, a user may interact with the user computing device 102 by interacting with the head unit controls. In addition, the vehicle 172 may provide a communication link 140, and the communication link 140 may, for example, include a wired connection to the vehicle 172 (e.g., via a USB connection), through which the user computing device 102 may transmit navigation information and corresponding navigation instructions for presentation within the instrument panel display unit 151, the display 176 and / or as an audio output through the speaker 184.
[0052] Therefore, the host unit 174 may include a display 176 for outputting navigation information such as a digital map. Of course, the dashboard display unit 151 may also display such navigation information, including a digital map. Such a map presented in the dashboard display unit 151 may provide a driver of the vehicle 172 with better positioned navigation instructions, and therefore, the driver may not be forced to remove his or her eyes from the active road as much while driving in order to safely navigate to his or her intended destination. However, in some implementations, the display 176 includes a software keyboard for inputting text input, which may include the name or address of the destination, departure point, etc.
[0053] The hardware input controls 178 and 180 on the head unit 174 and the steering wheel, respectively, can be used to enter alphanumeric characters or perform other functions for requesting navigation guidance. For example, the hardware input controls 178, 180 can be and / or include rotary controls (e.g., knobs), touch pads, touch screens, and / or any other suitable input controls. For example, the head unit 174 can also include audio input and output components, such as a microphone 182 and a speaker 184. As an example, the user computing device 102 can be communicatively connected to the head unit 174 (e.g., via Bluetooth™, WiFi, cellular communication protocols, wired connections, etc.) or can be included in the head unit 174. The user computing device 102 can present map information via the dashboard display unit 151, transmit audio instructions for navigation via the speaker 184, and receive input from the user via the head unit 174 (e.g., via the user interacting with the input controls 178 and 180, the display 176, or the microphone 182).
[0054] Example dialogue / analysis performed to determine refined navigation search results
[0055] Reference below FIG. 2A to FIG. 2C The dialog flow and processing workflow shown are used to discuss the technology of the present disclosure for determining routes and locations through natural dialogue. FIG. 2A to FIG. 2C In some implementations, the actions described as being performed by the user computing device 102 may be performed by the external server 120, the vehicle computing device 150, and / or may be performed in parallel by the user computing device 102, the navigation server 120, and / or the vehicle computing device 150. For example, the user computing device 102, the navigation server 120, and / or the vehicle computing device 150 may utilize the language processing modules 109a, 120a, 153a and / or the machine learning modules 109b, 120c, 153c to determine routes and locations through natural conversations with the user.
[0056] Specifically, Figure 2A shows that user 202 and Figure 1A200 between the user computing device 102 of the embodiment of the present invention, so as to determine the place and route through natural dialogue. The user 202 can audibly talk with the user computing device 102, which can prompt the user 202 to clarify, so as to determine a refined navigation search result set, which enables the user 202 to go to the desired destination of the user 202. That is, the user 202 can provide user input (transmission to the user computing device 102 as shown in 204a) to the user computing device 102. The user input can generally include the desired destination of the user 202, and the additional criteria related to the desired route of the user 202 that the user 202 includes to arrive at the destination. For example, the user 202 may say "Navigate to the ABC hotel (navigation to the ABC hotel)", and the user 202 may also say "I do not want to drive for longer than 25 minutes (I do not want to drive more than 25 minutes)". Therefore, the user input includes the destination (ABC hotel) and the additional criteria (travel time is less than or equal to 25 minutes).
[0057] Using this user input, the user computing device 102 may generate an initial set of navigation search results that meets one or both of the criteria of the user 202. For example, the initial set of navigation search results may include multiple routes to one or more ABC hotels, and / or multiple routes to different hotels / accommodations that are less than or equal to 25 minutes away. Regardless, the user computing device 102 may determine that the number of candidate routes is too large to provide to the user 202, and / or may otherwise determine that the set of navigation search results should be filtered in order to provide the user 202 with a refined set of navigation search results.
[0058] In this case, the user computing device 102 may generate an audio request that is output to the user 202 via a speaker 206 that may be integrated as part of the user computing device 102 (e.g., part of the I / O module 118) (transmission to the user 202 as shown at 204b). The audio request may prompt the user 202 to provide additional criteria and / or details corresponding to the desired destination and / or route of the user 202 so that the user computing device 102 (e.g., via the machine learning module 109b) refines the navigation search result set. Continuing with the above example, the audio request transmitted to the user 202 via the speaker 206 may state “What is the address of the ABC hotel where you are staying?”, and the audio request may further state “Several routes include traveling on toll roads. Is that okay?” In this manner, the user computing device 102 may request additional information from the user 202 in order to filter (eg, delete) routes that do not meet and / or fail to satisfy additional criteria that the user 202 may provide in response to the audio request.
[0059] However, the audio request may provide the user with a series of various clarifying options. For example, user 202 may be traveling in Switzerland and may provide user input by saying "navigate to a nearby hiking spot". User computing device 102 may generate a large number of route options in the navigation search result set, including several different options for reaching a hiking trail from a car parking lot, and device 102 may respond to user 202 with an audio request stating "Some of the top rated options require taking a cable car from the parking lot, would you be willing to do that? The total journey time would likely be under 30 minutes. (Some of the top rated options require taking a cable car from the parking lot, would you be willing to do that? The total journey time may be less than 30 minutes.)" Therefore, user computing device 102 may provide user 202 with an audio request that may quickly delete many route options based on user 202's response, which indicates whether taking a cable car from the parking lot is acceptable.
[0060] As another example, user computing device 102 may provide an audio request, which includes suggestions for helping user 202 decide the best route in this navigation search result set. In this example, user 202 may arrive at the airport, and may want to navigate to their hotel by asking user computing device 102 "Give me directions to ABC hotel (guide to ABC hotel)". User computing device 102 may respond to several different route suggestions and the best candidate, stating "The route I recommend is the shortest one, but it involvesdriving 10 miles on a single track road. (The route recommended for you is the shortest route, but it requires driving 10 miles on a single track.)" If user 202 is willing to drive 10 miles on a single track, user 202 may accept the route of suggestion, thereby ending route search. However, if user 202 refuses the suggested route, user computing device 102 may delete the route of suggestion and include all routes of driving at least 10 miles on a single track. Thus, proposing suggested routes using specified criteria may enable the user computing device 102 to refine the set of navigation search results without directly prompting (and potentially distracting) the user 202 .
[0061] As another example, an audio request may be configured to provide conversation clarification during a navigation session. Similar to the above example, user 202 may be navigating from an airport to their hotel, and while user 202 is en route, user 202 encounters a potential detour route along the way that will take a similar amount of time but has different characteristics. When approaching the detour route, user computing device 102 may prompt user 202 with an audio request stating, "There is an alternateroute at exit 213A with a similar ETA. It is a shorter distance but has some temporary construction which may cause a 3-minute delay. Would you like to take the alternate route?" In response, user 202 may accept or reject the alternate route, and user computing device 102 may continue with the original route or switch to the alternate route as needed. Thus, in this example, the navigation search result set may include the original route and the alternate route, and the audio request may prompt the user 202 to filter the navigation search result set by determining which of the two routes the user 202 prefers. In this manner, the audio request provided by the user computing device 102 may proactively / continuously search and / or filter the navigation search result set before / during the navigation session to ensure that the user 202 receives the best route experience to their destination.
[0062] Additionally, it should be noted that the user computing device 102 may typically allow the user 202 several seconds (e.g., 5 to 10 seconds) to respond after the audio request is transmitted through the speaker 206, so as to give the user 202 sufficient time to think of an appropriate response without constantly listening to the interior of the vehicle. By default, the user computing device 102 may not activate a microphone and / or other listening device (e.g., included as part of the I / O module 118) while running the navigation app 108 and / or while processing information received through the microphone by, for example, the processor 104, the language processing module 109a, the machine learning module 109b, and / or the OS 110, or in accordance with, for example, the processor 104, the language processing module 109a, the machine learning module 109b, and / or the OS 110. Thus, the user computing device 102 may not actively listen to the interior of the vehicle during a navigation session and / or at any other time, unless the user computing device 102 provides the user 202 with an audio request, the user computing device 102 may expect the user 202 to verbally respond to the audio request within a few seconds of transmission.
[0063] In any case, user 202 can hear the audio request, and in response, can provide subsequent user input (transmission to user computing device 102 as shown in 204c). Subsequent user input can generally include additional route / destination criteria based on the requested information that includes a portion of the audio request provided for user computing device 102. Continuing the previous example, user 202 may provide subsequent user input to the audio request "What is the address of the ABC hotel where you are staying?" and "Several routes include traveling on toll roads. Is that okay? " by saying "The ABC hotel is at 123 Main Street, Chicago, IL" and "No, I would prefer to avoid toll roads. (No, I would rather avoid toll roads.)" Thus, in this example, the user 202 provides additional location information and route information related to the desired destination to exclude toll roads that the user computing device 102 can use to optimize the set of navigation search results. Accordingly, the user computing device 102 can receive subsequent user input and can continue to generate a refined set of navigation search results. The user computing device 102 can provide the refined set of navigation search results to the user 202 as an audio output (e.g., through the speaker 206), a visual output on a display screen (e.g., the instrument panel display unit 151, the display 176), and / or a combination of audio / visual output.
[0064] To better understand Figure 2A The processing performed by the user computing device 102 described in Figure 2B A user input analysis sequence 210 is shown in order to output an audio request and a navigation search result set. The user input analysis sequence 210 generally includes the user computing device 102 analyzing / manipulating the user input during two different time periods 212, 214 to generate two different outputs. That is, during a first time period 212, the user computing device 102 receives the user input and continues to utilize the language processing module 109a to generate a text transcription of the user input. Thereafter, during a second time period 214, the user computing device utilizes the language processing module 109a and / or the machine learning module 109b to analyze the text transcription of the user input in order to output an audio request and / or a navigation search result set.
[0065] More specifically, during the first time period 212, the user computing device 102 receives user input via an input device (e.g., a microphone as part of the I / O module 118). The user computing device 102 then utilizes the processor 104 to execute instructions included as part of the language processing module 109a to transcribe the user input into a text set. The user computing device 102 may cause the processor 104 to execute instructions including, for example, an ASR engine (e.g., ASR engine 109a1) to transcribe the user input from the speech-based input received by the I / O module 118 into a text transcript of the user input. Of course, as previously described, it should be understood that executing the ASR engine to transcribe the user input into a text transcript (and reference to the ASR engine 109a1) is not limited to the text transcript of the user input. Figure 2B and Figure 2C and any other actions described) may be performed by the user computing device 102, the external server 120, the vehicle computing device 150, and / or any other suitable component or combination thereof.
[0066] This transcription of the user input may then be analyzed during a second period 214, for example, by a processor 104 executing instructions including a language processing module 109a and / or a machine learning module 109b, to output an audio request and / or a set of navigation search results. Specifically, instructions including the language processing module 109a and / or the machine learning module 109b may cause the processor 104 to interpret the text transcription to determine the user intent and values corresponding to the destination and / or other constraints. For example, the user intent may include traveling to a desired destination, the destination value may correspond to a specific location (e.g., Chicago, Illinois) or a general location (e.g., a nearby hiking trail), and the other constraints may include any other details corresponding to the user intent (e.g., traveling "by car," "less than 10 miles," etc.).
[0067] To determine a destination value based on the user's input when a destination is generally described (e.g., "nearby restaurants"), the user computing device 102 may first parse and extract this destination information from the user's input. The user computing device 102 may then access a database (e.g., a map database 156, a POI database 159) or other suitable repository to search for corresponding locations by anchoring the search on the user's current location and / or viewport. The user computing device 102 may then identify candidate destinations and routes to each candidate destination based on similarities between the locations in the repository and the destinations determined based on the user's input, thereby creating an initial set of navigation search results.
[0068] However, before determining whether to generate an audio request, the user computing device 102 can prune this initial set of navigation search results by removing candidate destinations and routes that do not match and / or otherwise correctly correspond to other details corresponding to the user's intent. For example, if a candidate destination is farther than a maximum distance specified by the user in the user input, the candidate destination can be removed from the initial set of navigation search results. In addition, each destination / route can receive a score corresponding to, for example, the overall similarity of the destination / route to the value extracted from the user input.
[0069] As the user computing device 102 determines and filters / prunes the initial set of navigation search results to generate the set of navigation search results, the device 102 may proceed to determine whether to provide audio output to the user. The user computing device 102 may make this determination based on a number of criteria, such as (i) the total number of routes / destinations that will be provided to the user as part of the set of navigation search results, (ii) the type of device and / or surface type (e.g., smartphone, tablet, wearable device, etc.) that the user is using to receive navigation instructions, (iii) the entry point and / or input type that the user is using to enter user input (e.g., voice-based input, touch-based input), (iv) whether the scores corresponding to the destinations / routes included in the set of navigation results are sufficiently high (e.g., relative to a score threshold), and / or any other suitable determination criteria or combination thereof.
[0070] For example, the user computing device 102 may determine that the total number of routes included as part of the navigation search result set is twenty, and the route presentation threshold may be fifteen. In some examples, the route presentation threshold is set based on a determination of the computational expense involved in providing the result set. For example, in this example, providing a result of sixteen results exceeds the threshold, and more computing resources are required to provide this result set than a result set that is less than the threshold amount. Therefore, the user computing device 102 compares the total number of routes to the route presentation threshold to determine that the total number of routes does not meet the route presentation threshold and an audio request should be generated. Therefore, if the user computing device 102 applies any of the above criteria, and any of the applied criteria fails to meet its corresponding threshold (e.g., route presentation threshold, score threshold) and / or has a corresponding value that requires an audio request (e.g., device type, input type), the device 102 may generate an audio request.
[0071] In response to determining that an audio request should be generated, the user computing device 102 may continue to generate the audio request using, for example, the language processing module 109a. The user computing device 102 may generally continue to generate the audio request by considering which audio request will minimize the number of destinations / routes included in the set of navigation search results. That is, the user computing device 102 may analyze the attributes corresponding to each destination / route, determine which attributes are most common among the destinations / routes included in the set of navigation search results, and may generate the audio request based on one or more of these most common attributes.
[0072] For example, a navigation search result set may include twenty route options to a particular destination, and each route option may differ from each other route option primarily in the distance to travel to the particular destination. Therefore, the user computing device 102 may generate an audio request prompting the user to provide a distance requirement in order to most efficiently refine the navigation search result set by removing routes that fail to meet the user's distance requirement.
[0073] As another example, a navigation search result set may include eight route options to a particular destination, and each route option may differ from each other route option primarily in the type of road (e.g., freeways, rural roads, scenic routes, city streets) that a user may travel to reach the particular destination. Accordingly, the user computing device 102 may generate an audio request prompting the user to provide a road type preference in order to most efficiently refine the navigation search result set by removing routes that fail to satisfy the user's road type preference.
[0074] The user computing device 102 may generate the text of the audio request by utilizing the language processing module 109a and, in some aspects, utilizing a large language model (LLM) (e.g., Language Model for Conversational Applications (LaMDA)) (not shown) included as part of the language processing module 109a. Such an LLM may be tuned / trained to generate the audio request text based on certain most common attributes of a set of navigation search results, and / or the LLM may be trained to receive as input a natural language representation of candidate routes / destinations and output a set of text representing the audio request based on the most common attributes.
[0075] In any case, when the user computing device 102 completely generates the text of the audio request, the device 102 can continue to synthesize the text into speech so as to provide the requested audio output to the user. Specifically, the user computing device 102 can transmit the text of the audio output to a TTS engine (e.g., TTS engine 109a2) so that the audio request can be audibly output through a speaker (e.g., speaker 206) so that the user can hear and interpret the audio output. In addition or alternatively, the user computing device 102 can also visually prompt the user by displaying the text of the audio request on a display screen (e.g., dashboard display unit 151, display 176) so that the user can interact with the display screen (e.g., click, tap, slide, etc.) and / or verbally respond to the audio request.
[0076] When the user receives the audio request from the user computing device 102, the user may provide subsequent user input. The user computing device 102 may receive the subsequent user input and continue to refine the navigation search result set, such as Figure 2C More specifically, Figure 2C A subsequent user input analysis sequence 220 is shown to output a refined set of navigation search results. The subsequent user input analysis sequence 220 generally includes the user computing device 102 analyzing / manipulating the subsequent user input during two different time periods 222, 224 to generate two different outputs. That is, during the first time period 222, the user computing device 102 receives the subsequent user input and continues to utilize the language processing module 109a to generate a text transcription of the subsequent user input. Thereafter, during the second time period 224, the user computing device utilizes the language processing module 109a and / or the machine learning module 109b to analyze the text transcription of the subsequent user input to output a refined set of navigation search results.
[0077] More specifically, during the first period 222, the user computing device 102 receives user input via an input device (e.g., a microphone as part of the I / O module 118). The user computing device 102 then uses the processor 104 to execute instructions included as part of the language processing module 109a to transcribe subsequent user input into a text set. The user computing device 102 may cause the processor 104 to execute instructions including, for example, an ASR engine (e.g., ASR engine 109a1) to transcribe subsequent user input from the speech-based input received by the I / O module 118 into a text transcription of the subsequent user input.
[0078] This transcription of the subsequent user input may then be analyzed during a second period 224, for example by a processor 104 executing instructions including a language processing module 109a and / or a machine learning module 109b, to output a refined set of navigation search results. Specifically, instructions including the language processing module 109a and / or the machine learning module 109b may cause the processor 104 to interpret the text transcription of the subsequent user input to determine a subsequent user intent and values corresponding to a refined destination value and / or other constraints. For example, the subsequent user intent may include determining whether the subsequent user input is related to an audio request, the refined destination value may correspond to a specific location (e.g., Chicago, Illinois) or a general location (e.g., a nearby hiking trail), and the other constraints may include any other details corresponding to the subsequent user intent (e.g., traveling "by car", "less than 10 miles", etc.).
[0079] As the user computing device 102 receives subsequent user input and determines subsequent user intent and refined destination values and / or other constraints, the device 102 may refine / filter the navigation search result set by removing candidate destinations and routes that do not match and / or otherwise correctly correspond to other details corresponding to the subsequent user intent, refined destination values, and / or other constraints. In addition, each destination / route included in the navigation search result set may receive a score (e.g., from the machine learning module 109b) corresponding to, for example, the overall similarity of the destination / route to the value extracted from the subsequent user input. For example, if a candidate route receives a score of 35 due to being relatively dissimilar to the value extracted from the subsequent user input, and the score threshold for retaining as part of the navigation result set is 75, then the candidate route may be deleted from the navigation search result set.
[0080] Generally speaking, the user computing device 102 can repeat the referenced Figure 2B and Figure 2C The user computing device 102 may perform the actions described above any appropriate number of times to provide a refined set of navigation search results to the user. For example, after receiving subsequent user input, the user computing device 102 may determine that subsequent audio output should be provided to the user. Therefore, in this example, the user computing device 102 may continue to generate subsequent audio requests for the user, as described above with reference to Figure 2B The user computing device 102 may then receive yet another user input in response to a subsequent audio request, and may continue to further refine the set of navigation search results until the criteria used by the device 102 to determine whether to generate an audio request are met.
[0081] Regardless, when the user computing device 102 determines that all criteria corresponding to generating an audio request are met, the device 102 may determine that the navigation search result set is a refined navigation search result set suitable for providing to the user. Therefore, the user computing device 102 may proceed to provide the refined navigation search result set to the user as an audio output and / or a visual display. When provided to the user, the refined navigation search result set may include any suitable information corresponding to the corresponding route, such as the total driving distance, the total driving time, the number of road changes / turns, and / or any other suitable information or combination thereof. In addition, all information included as part of each route of the refined navigation search result set may be provided to the user as an audio output (e.g., via the speaker 206) and / or as a visual display on a display screen of any suitable device (e.g., the I / O module 118, the instrument panel display unit 151, the display 176).
[0082] Of course, the user may determine that the set of navigation search results should be further refined, and may independently provide input to the user computing device 102 (e.g., without prompting from the user computing device 102) to achieve this. In some aspects, the user may provide a particular trigger phrase or word to the user input that causes the user computing device 102 to receive user input for a certain duration following the user input with the trigger phrase / word. The user may initialize the set of inputs to the user computing device 102 in this or a similar manner, and the device 102 may continue to refine the set of inputs in a manner consistent with the previously referenced search results. FIG. 2A to FIG. 2C The user input may be received and interpreted in a similar manner as described above. For example, the user may separately say “I'd prefer a shorter distance route, and am willing to drive on a single track road.” The user computing device 102 may receive this user input and may continue to refine the navigation search result set to provide to the user, as described above.
[0083] Example dialogue / analysis performed to provide navigation instructions
[0084] When the user computing device 102 has successfully generated a refined set of navigation search results, the user can review the results to determine the best route to the desired destination. To illustrate the operations performed by the user computing device 102 as part of the route acceptance process, Figure 3A and Figure 3BAn example route acceptance and route adjustment sequence is shown in which a user provides input to select / adjust a route included as part of a refined set of navigation search results. As previously described, each route included as part of a refined set of navigation search results includes turn-by-turn navigation instructions to a destination as part of a navigation session. As described herein, a user may provide input regarding acceptance and / or adjustment of a route included in a refined set of navigation search results, such that turn-by-turn navigation instructions provided by the user computing device 102 during a navigation session may also change in response to the user's input regarding a currently accepted route.
[0085] More specifically, Figure 3A An example transition 300 between a user 202 providing a route acceptance input and a user computing device 102 displaying navigation instructions corresponding to the accepted route is shown. The user 202 may provide a route acceptance input to the user computing device 102 indicating acceptance of a route included as part of a refined set of navigation search results. The user computing device 102 may then receive the route acceptance input and proceed to initiate a navigation session that includes turn-by-turn navigation instructions corresponding to the accepted route. Thus, the user computing device 102 may continue to provide verbal turn-by-turn navigation instructions to the user 202, as well as present turn-by-turn navigation instructions on a display screen 302 of the device 102 for the user 202 to view.
[0086] During the navigation session, the user computing device 102 may display, via the display screen 302, a map depicting the location of the user computing device 102, the heading of the user computing device 102, an estimated time of arrival, an estimated distance to the destination, an estimated travel time to the destination, current navigation directions, one or more upcoming navigation directions corresponding to the set of navigation instructions for the accepted route, one or more user-selectable options for changing the display or adjusting the navigation directions, etc. The user computing device 102 may also issue audio instructions corresponding to the set of navigation instructions.
[0087] As an example, the user computing device 102 may provide the user 202 with a refined set of navigation search results that includes three candidate routes to the desired destination of the user 202. The user 202 may provide a route acceptance input indicating that the user 202 wishes to take a first candidate route included as part of the refined set of navigation search results. The user computing device 102 may receive this route acceptance input from the user 202 and may proceed to provide a first navigation instruction included as part of the first candidate route (referred to herein as the "accepted route" in this example) and present a map including a visual representation of the first navigation instruction on the display screen 302. As the user 202 travels along the accepted route, the user computing device 102 may verbally and visually provide sequential navigation instructions (e.g., first, second, third) to the user 202 as the user 202 approaches each waypoint along the accepted route to enable the user 202 to follow the accepted route. When the user 202 arrives at the destination at the end of the accepted route, the user computing device 102 may deactivate the navigation session.
[0088] However, in certain circumstances, user 202 may wish and / or be forced to change from an accepted route to an alternate route. Figure 3B An example route update sequence 320 is shown to update the navigation instructions provided to the user 202 by prompting the user 202 with the option to switch to an alternate route. Specifically, when the user computing device 102 determines that the alternate route may be a better route than the accepted route, the user computing device 102 may actively participate in the navigation session initiated by the user 202. The user computing device 102 may make such a determination based on, for example, updated traffic information along the accepted route (e.g., from the traffic database 157) and / or any other suitable information.
[0089] Based on this determination, the user computing device 102 may generate an alternative route output that provides the user 202 (indicated by the transmission to the user 202 at 322a) with the option of adjusting the current navigation session to follow the alternative route. For example, the user computing device 102 may verbally provide the alternative route output via the speaker 206 and / or may visually indicate the alternative route output via the prompt 324. Figure 3BAs shown, the alternate route output may state “There is an Alternate Route that decreases travel time by 10 minutes. Would you like to switch to the Alternate Route?” This wording may be provided to the user 202 verbally via the speaker 206 or may be visually presented via the display screen 322.
[0090] If the user 202 decides to provide verbal user input (transmission to the user computing device 102 indicated by 322 b), the user 202 may verbally respond to the alternative route output within a short period of time (e.g., 5 to 10 seconds) after the output is provided to the user 202 so that the user computing device 102 receives the verbal user input. The user computing device 102 may receive the verbal user input and may respond similarly to the embodiments described herein with reference to FIG. 2A to FIG. 2C The analysis described above continues to process / analyze the verbal user input. Specifically, if the user 202 decides to accept the alternative route, the user computing device 102 may initiate an updated navigation session to provide alternative turn-by-turn navigation instructions based on the alternative route. Alternatively, if the user 202 decides to reject the alternative route, the user computing device 102 may continue to provide turn-by-turn navigation instructions corresponding to the accepted route and may not initiate an updated navigation session.
[0091] In addition, the visual presentation of the alternative route output may include interactive buttons 324a, 324b that enable the user 202 to physically interact with the display screen 322 to accept or decline the switch to the alternative route. When the user receives the prompt 324, the user may interact with the prompt 324 by pressing, clicking, tapping, sliding, etc., one of the interactive buttons 324a, 324b. If the user selects the "yes" interactive button 324a, the user computing device 102 may instruct the navigation application 108 to generate and present turn-by-turn navigation directions as part of an updated navigation session corresponding to the alternative route. If the user selects the "no" interactive button 324b, the user computing device 102 may continue to generate and present turn-by-turn navigation instructions corresponding to the accepted route, and may not generate / present an updated navigation session.
[0092] Example logic for determining location and directions through natural conversation
[0093] Figure 41 is a flow chart of an example method 400 for determining a location and route through natural conversation, which may be implemented in a computing device (such as the user computing device 102 of FIG. 1 ). It should be understood that for ease of discussion only, reference is made herein to Figure 4 The “user computing device” discussed herein may correspond to user computing device 102. Furthermore, it should be understood that in Figure 4 Throughout the description of FIG. 1 , in some implementations, actions described as being performed by the user computing device 102 may be performed by an external server 120, a vehicle computing device 150, and / or may be performed in parallel by the user computing device 102, the navigation server 120, and / or the vehicle computing device 150. For example, the user computing device 102, the navigation server 120, and / or the vehicle computing device 150 may utilize the language processing modules 109a, 120a, 153a and / or the machine learning modules 109b, 120c, 153c to determine routes and locations through natural conversations with the user.
[0094] Go to Figure 4 , method 400 may be implemented by a user computing device (e.g., user computing device 102). Method 400 may be implemented in an instruction set stored on a computer readable memory and executable at one or more processors (e.g., processor 104) of the user computing device.
[0095] At box 402, method 400 includes receiving a voice input including a search query from a user to initiate a navigation session (box 402). Method 400 may further include an optional step of transcribing the voice input into a text collection (box 404). In some aspects, method 400 may further include parsing the text collection by one or more processors to determine a destination value, and extracting the destination value from the text collection. In addition, in these aspects, method 400 may include searching for the destination value in a destination database (e.g., map database 156, external server 120), and identifying multiple destinations based on the results of searching the destination database. Therefore, in these aspects, method 400 may further include generating one or more routes to each of the multiple destinations.
[0096] The method 400 also includes generating a navigation search result set in response to the search query (block 406). The navigation search result set may include multiple destinations or multiple routes corresponding to multiple destinations. In some aspects, generating the navigation search result set in response to the search query further includes transcribing the speech input into a text set, and applying a machine learning (ML) model to the text set to output the user intent and the destination. In these aspects, the ML model may be trained using one or more text training data sets to output one or more training intents and one or more training destinations.
[0097] In some aspects, generating a set of navigation search results in response to a search query further includes generating one or more candidate routes to each of a plurality of destinations based on a corresponding attribute set for each of the one or more candidate routes. In these aspects, each corresponding attribute set may include one or more of: (i) mode of transportation, (ii) number of transfers, (iii) total driving distance, (iv) total driving time, (v) total driving distance on each included road, or (vi) total driving time on each included road.
[0098] The method 400 further includes providing an audio request to the user for refining the set of navigation search results (block 408). In some aspects, the method 400 may further include determining whether to provide the audio request to the user based on at least one of: (i) a total number of routes included in the plurality of routes, (ii) a device type of a device used by the user to provide voice input, (iii) a type of input provided by the user, or (iv) a number of second routes included in the plurality of routes that meet a quality threshold.
[0099] In some aspects, method 400 may include verbally transmitting the audio request for consideration by a text-to-speech (TTS) engine (e.g., TTS engine 109a2). In addition, in some aspects, providing the user with the audio request for refining the set of navigation search results further includes determining a primary attribute of the plurality of routes that will result in minimizing the plurality of routes, and generating the audio request for the user based on the primary attribute. In some aspects, providing the user with the audio request for refining the set of navigation search results may further include generating the audio request based on the attributes of the plurality of routes by executing a large language model (LLM).
[0100] The method 400 further includes receiving, in response to the audio request, a subsequent voice input from the user including the refined search query (block 410). In some aspects, the method 400 may further include recognizing the voice input and the subsequent voice input based on a trigger phrase included as part of both the voice input and the subsequent voice input.
[0101] Method 400 may further include an optional step of filtering the set of navigation search results based on subsequent user input (block 412). That is, in some aspects, method 400 may further include transcribing the speech input into a text set, (a) providing an audio request to the user for refining the set of navigation search results, (b) receiving a subsequent speech input from the user including a refined search query in response to the audio request, and (c) filtering the set of navigation search results to generate one or more refined navigation search results by removing routes from the plurality of routes based on the subsequent speech input. In some aspects, filtering the set of navigation search results to generate one or more refined navigation search results further includes removing routes from the set of routes having corresponding relevance scores that do not meet a relevance threshold based on a natural language transcription of the subsequent speech input by executing a machine learning (ML) model.
[0102] In addition, in these aspects, the natural language transcription may not be parsed, and the ML model may be configured to receive the natural language transcription and the route as input to output a relevance score for each route. Instead, the ML model may be trained using the speech input and the transcribed string of the training route to output a relevance score corresponding to each responsive training route. The relevance score may generally indicate the relevance of a particular route based on the transcribed string of the user input. In this way, the ML model may operate on a more "end-to-end" basis by not parsing the user input to extract explicit attributes, but determining the relevance score for each route based on the user's input. For example, the ML model may receive a natural language transcription of a subsequent user input stating "I'd prefer nosingle track roads (I don't like single-lane roads)", and two routes from a route set as input. The first route may include navigation instructions that guide the user along a series of one-way roads, while the second route may include navigation instructions that guide the user not along one-way roads.
[0103] Continuing with the above example, the ML model may output relevance scores for two routes, which may indicate relevance as an indicator of route feasibility or route infeasibility. That is, the ML model may output a relatively high relevance score (e.g., 9 out of 10) for the first route because the first route includes a series of one-way streets, and the ML model may output a relatively low relevance score (e.g., 1 out of 10) for the second route because the second route does not include one-way streets. In this way, the relevance scores may indicate route infeasibility because the first route has a high relevance score based on the first route including a series of one-way streets (which the user does not want), while the second route has a low relevance score based on the second route not including one-way roads (which the user prefers). Alternatively, the ML model may output a relatively low relevance score (e.g., 1 out of 10) for the first route because the first route includes a series of one-way streets, and the ML model may output a relatively high relevance score (e.g., 9 out of 10) for the second route because the second route does not include one-way streets. In this way, the relevance scores may indicate route feasibility because a first route has a low relevance score based on the first route including a series of one-way roads (which the user does not want), while a second route has a high relevance score based on the second route not including one-way roads (which the user prefers).
[0104] The method 400 may also include an optional step of determining whether to provide a subsequent audio request to the user based on one or more refined navigation search results (box 414). Specifically, optionally, the user computing device 102 may determine whether the navigation search result set meets the route presentation threshold (box 416). If the user computing device 102 determines that the navigation search result set does not meet the route presentation threshold (the "no" branch of box 416), the method 400 may return to box 408, where the user computing device 102 provides the subsequent audio request to the user. However, if the user computing device 102 determines that the navigation search result set does meet the route presentation threshold (the "yes" branch of box 416), the method 400 may continue to box 418. It should be understood that the method 400 may include iteratively executing each of boxes 408 to 416 (and / or any other box of the method 400) any appropriate number of times until one or more refined navigation search results meet the route presentation threshold.
[0105] Regardless, method 400 further includes providing one or more refined navigation search results that include a subset of the plurality of destinations or the plurality of routes in response to the refined search query (block 418). In some aspects, method 400 may further include providing the one or more refined navigation search results at a user interface for viewing by a user.
[0106] In some aspects, method 400 may include receiving verbal route acceptance input from a user indicating an accepted route from one or more refined navigation search results. In these aspects, method 400 may further include displaying the accepted route at a user interface for the user to view, and initiating a navigation session along the accepted route by providing verbal navigation instructions corresponding to the accepted route while the user is traveling along the accepted route.
[0107] In some aspects, providing one or more refined navigation search results responsive to the refined search query further includes generating a text summary for each route in the subset of the plurality of routes by executing a large language model (LLM). Additionally, in these aspects, method 400 may include providing the subset of the plurality of routes and each corresponding text summary at a user interface for viewing by a user.
[0108] In certain aspects, method 400 may further include receiving a selection of the accepted route from the user to initiate a navigation session along the accepted route. Additionally, in these aspects, method 400 may include determining during the navigation session that an alternate route improves one of (i) a user arrival time, (ii) a user travel distance, or (iii) a user's time on a particular road. Method 400 may also include prompting the user during the navigation session, via a verbal prompt or a text prompt, an option to switch from the selected route to the alternate route.
[0109] Aspects of the Disclosure
[0110] 1. A method in a computing device for determining a location and a route through natural conversation, the method comprising: receiving a voice input including a search query from a user to initiate a navigation session; generating, by one or more processors, a navigation search result set in response to the search query, the navigation search result set including a plurality of destinations and / or a plurality of routes corresponding to one or more destinations; providing, by the one or more processors, an audio request to the user for refining the navigation search result set; receiving, in response to the audio request, a subsequent voice input including a refined search query from the user; and providing, by the one or more processors, one or more refined navigation search results including a subset of the plurality of destinations or the plurality of routes in response to the refined search query.
[0111] 2. The method as described in aspect 1 further includes: transcribing the voice input into a text set by an automatic speech recognition (ASR) engine; (a) providing the audio request for refining the navigation search result set to the user by the one or more processors; (b) receiving the subsequent voice input including the refined search query from the user in response to the audio request; (c) filtering the navigation search result set by the one or more processors to generate the one or more refined navigation search results by deleting routes from the multiple routes based on the subsequent voice input; (d) determining by the one or more processors whether to provide a subsequent audio request to the user based on the one or more refined navigation search results; and (e) iteratively performing (a) to (e) until the one or more refined navigation search results meet a threshold.
[0112] 3. A method as described in aspect 2, wherein filtering the navigation search result set to generate the one or more refined navigation search results further includes: the one or more processors executing a machine learning (ML) model remove routes in the route set having corresponding relevance scores that do not meet a relevance threshold based on a natural language transcription of the subsequent voice input, wherein the natural language transcription is not parsed, and the ML model is configured to receive the natural language transcription and the route as input to output a relevance score for each route.
[0113] 4. The method as described in any one of aspects 1 to 3 further comprises: providing the one or more refined navigation search results at a user interface for user viewing.
[0114] 5. The method as described in any one of Aspects 1 to 4 further includes: determining, by the one or more processors, whether to provide the audio request to the user based on at least one of: (i) the total number of routes included in the multiple routes, (ii) the device type of the device used by the user to provide the voice input, (iii) the input type provided by the user, or (iv) the number of second routes included in the multiple routes that meet the quality threshold.
[0115] 6. The method of any one of aspects 1 to 5, further comprising: verbally transmitting the audio request via a text-to-speech (TTS) engine for consideration by the user.
[0116] 7. The method as described in any one of aspects 1 to 6 further includes: receiving verbal route acceptance input from the user, the verbal route acceptance input indicating an accepted route from the one or more refined navigation search results; displaying the accepted route at the user interface for the user to view; and when the user travels along the accepted route, initiating the navigation session along the accepted route by the one or more processors by providing verbal navigation instructions corresponding to the accepted route.
[0117] 8. A method as described in any of Aspects 1 to 7, wherein generating the navigation search result set in response to the search query further includes: transcribing the voice input into a text set; and applying a machine learning (ML) model to the text set by the one or more processors to output user intent and a destination, wherein the ML model is trained using one or more text training data sets to output one or more training intents and one or more training destinations.
[0118] 9. The method as described in any one of Aspects 1 to 8 further includes: transcribing the speech input into a text collection; parsing the text collection by the one or more processors to determine a destination value; extracting the destination value from the text collection by the one or more processors; and searching the destination value in a destination database by the one or more processors.
[0119] 10. The method of aspect 9, further comprising: identifying, by the one or more processors, the plurality of destinations based on a result of searching the destination database; and generating, by the one or more processors, one or more routes to each of the plurality of destinations.
[0120] 11. A method as described in any one of aspects 1 to 10, wherein generating the navigation search result set in response to the search query further includes: generating, by the one or more processors, the one or more candidate routes to each of the multiple destinations based on a corresponding attribute set of each of the one or more candidate routes, wherein each corresponding attribute set includes one or more of the following: (i) mode of transportation, (ii) number of transfers, (iii) total driving distance, (iv) total driving time, (v) total driving distance on each included road, or (vi) total driving time on each included road.
[0121] 12. A method as described in any of Aspects 1 to 11, wherein providing the audio request to the user for refining the navigation search result set further includes: determining, by the one or more processors, a primary attribute of the multiple routes that will result in minimizing the multiple routes; and generating, by the one or more processors, the audio request for the user based on the primary attribute.
[0122] 13. The method of any one of aspects 1 to 12, wherein providing the audio request to the user for refining the navigation search result set further comprises: generating the audio request based on attributes of the plurality of routes by the one or more processors executing a large language model (LLM).
[0123] 14. A method as described in any of Aspects 1 to 13, wherein providing one or more refined navigation search results in response to the refined search query further includes: generating, by the one or more processors executing a large language model (LLM), a text summary for each route in the subset of the multiple routes; and providing the subset of the multiple routes and each corresponding text summary at the user interface for viewing by the user.
[0124] 15. The method of any one of aspects 1 to 14, further comprising: receiving a selection of an accepted route from the user to initiate the navigation session along the accepted route; determining during the navigation session that an alternate route improves one of (i) user arrival time, (ii) user travel distance, or (iii) user time on a particular road; and prompting the user with an option to switch from the selected route to the alternate route via a verbal prompt or a text prompt during the navigation session.
[0125] 16. The method of any one of aspects 1 to 15, further comprising: recognizing, by the one or more processors, the voice input and the subsequent voice input based on a trigger phrase that is part of both the voice input and the subsequent voice input.
[0126] 17. A computing device for determining locations and routes through natural conversation, the computing device comprising: a user interface; one or more processors; and a computer-readable memory, which is optionally non-transitory, coupled to the one or more processors and storing instructions thereon, which instructions, when executed by the one or more processors, cause the computing device to: receive voice input including a search query from a user to initiate a navigation session; generate a navigation search result set in response to the search query, the navigation search result set including multiple destinations or multiple routes corresponding to one or more destinations; provide an audio request to the user for refining the navigation search result set; receive subsequent voice input including a refined search query from the user in response to the audio request; and provide one or more refined navigation search results including a subset of the multiple destinations or the multiple routes in response to the refined search query.
[0127] 18. A computing device as described in aspect 17, wherein the instructions, when executed by the one or more processors, cause the computing device to: transcribe the speech input into a text set by an automatic speech recognition (ASR) engine; (a) provide the user with the audio request for refining the navigation search result set; (b) receive the subsequent speech input including the refined search query from the user in response to the audio request; (c) filter the navigation search result set by deleting routes from the multiple routes based on the subsequent speech input to generate the one or more refined navigation search results; (d) determine whether to provide a subsequent audio request to the user based on the one or more refined navigation search results; and (e) iteratively perform (a) to (e) until the one or more refined navigation search results meet a threshold.
[0128] 19. A computer-readable medium, which is optionally non-transitory, storing instructions for determining locations and routes through natural conversation, which instructions, when executed by one or more processors, cause the one or more processors to: receive voice input including a search query from a user to initiate a navigation session; generate a navigation search result set in response to the search query, the navigation search result set including multiple destinations or multiple routes corresponding to one or more destinations; provide an audio request to the user for refining the navigation search result set; in response to the audio request, receive subsequent voice input from the user including a refined search query; and provide one or more refined navigation search results including a subset of the multiple destinations or the multiple routes in response to the refined search query.
[0129] 20. A computer-readable medium as described in aspect 19, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: transcribe the speech input into a text set by an automatic speech recognition (ASR) engine; (a) provide the audio request for refining the navigation search result set to the user; (b) receive the subsequent speech input including the refined search query from the user in response to the audio request; (c) filter the navigation search result set by deleting routes from the multiple routes based on the subsequent speech input to generate the one or more refined navigation search results; (d) determine whether to provide a subsequent audio request to the user based on the one or more refined navigation search results; and (e) iteratively perform (a) to (e) until the one or more refined navigation search results meet a threshold.
[0130] 21. A computing device for determining location and route through natural conversation, the computing device comprising: a user interface; one or more processors; and a non-transitory computer readable memory, the non-transitory computer readable memory coupled to the one or more processors and storing instructions thereon, which instructions, when executed by the one or more processors, cause the computing device to perform any method disclosed herein.
[0131] 22. A tangible, non-transitory computer-readable medium storing instructions for determining a location and route through natural conversation, which instructions, when executed by one or more processors, cause the one or more processors to perform any of the methods disclosed herein.
[0132] 23. A method in a computing device for determining a location and a route through a natural conversation, the method comprising: receiving input from a user to initiate a navigation session; generating, by one or more processors, one or more destinations or one or more routes in response to the user input; providing, by the one or more processors, a request to the user to refine a response to the user input; receiving subsequent input from the user in response to the request; and providing, by the one or more processors, one or more updated destinations or one or more updated routes in response to the subsequent user input.
[0133] 24. The method of clause 23, wherein the user input is a voice input or a text input, and the request is an audio request or a text request.
[0134] Additional considerations
[0135] The following additional considerations apply to the foregoing discussion. Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Although the individual operations of one or more methods are shown and described as separate operations, one or more of the individual operations may be performed simultaneously, and the operations are not required to be performed in the order shown. The structures and functions presented as separate components in the example configuration may be implemented as combined structures or components. Similarly, the structures and functions presented as single components may be implemented as separate components. These and other variations, modifications, additions, and improvements all fall within the scope of the subject matter of the present disclosure.
[0136] In addition, some embodiments are described herein as including logic or multiple components, modules, or mechanisms. A module may constitute a software module (e.g., code stored on a machine-readable medium) or a hardware module. A hardware module is a tangible unit that is capable of performing certain operations and may be configured or arranged in a particular manner. In an example embodiment, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured as a hardware module by software (e.g., an application or an application portion), and the hardware module operates to perform certain operations as described herein.
[0137] In various embodiments, hardware modules may be implemented mechanically or electronically. For example, a hardware module may include dedicated circuitry or logic that is permanently configured (e.g., as a dedicated processor such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)) to perform certain operations. A hardware module may also include programmable logic or circuitry (e.g., contained within a general purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware module mechanically, in a dedicated and permanently configured circuit, or in a temporarily configured circuit (e.g., configured by software) may be driven by cost and time considerations.
[0138] Thus, the term "hardware" should be understood to encompass tangible entities, i.e., physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) entities, to operate in a particular manner or to perform certain operations described herein. As used herein, a "hardware-implemented module" refers to a hardware module. Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one moment in time. For example, where a hardware module includes a general-purpose processor configured using software, the general-purpose processor may be configured as correspondingly different hardware modules at different times. The software may configure the processor accordingly, e.g., to constitute a particular hardware module at one instance in time, and to constitute a different hardware module at a different moment in time.
[0139] Hardware modules can provide information to other hardware and receive information from other hardware. Therefore, the described hardware modules can be considered as communicatively coupled. In the case where multiple such hardware modules exist simultaneously, communication can be achieved by signal transmission (e.g., through appropriate circuits and buses) connecting the hardware modules. In embodiments where multiple hardware modules are configured or instantiated at different times, for example, communication between such hardware modules can be achieved by storing information in a memory structure accessible to multiple hardware modules and retrieving information from the memory structure. For example, a hardware module can perform an operation and store the output of the operation in a memory device communicatively coupled thereto. Another hardware module can then access the memory device at a later time to retrieve and process the stored output. The hardware module can also initiate communication with an input or output device, and can operate on a resource (e.g., a collection of information).
[0140] Method 400 may include one or more functional blocks, modules, separate functions or routines in the form of tangible computer executable instructions stored in a computer readable storage medium, optionally a non-transitory computer readable storage medium, and executed using a processor of a computing device (e.g., a server device, a personal computer, a smart phone, a tablet computer, a smart watch, a mobile computing device, or other client computing device, as described herein). Method 400 may be included as part of any back-end server (e.g., a map data server, a navigation server, or any other type of server computing device, as described herein), a client computing device module such as an example environment, or as part of a module external to such an environment. Although other figures may be described with reference to these figures for ease of explanation, method 400 may be utilized with other objects and user interfaces. In addition, although the above explanation describes the steps of method 400 as being performed by a particular device (such as a user computing device), this is done for illustrative purposes only. The blocks of method 400 may be performed by one or more devices or other parts of the environment.
[0141] The various operations of the example methods described herein may be performed, at least in part, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Such processors, whether temporarily configured or permanently configured, may constitute processor-implemented modules that operate to perform one or more operations or functions. In some example embodiments, the modules mentioned herein may include processor-implemented modules.
[0142] Similarly, the methods or routines described herein may be processor-implemented at least in part. For example, at least some of the operations of the method may be performed by one or more processors or processor-implemented hardware modules. The execution of certain operations may be distributed between one or more processors that not only reside in a single machine but also are deployed across multiple machines. In some example embodiments, one or more processors may be located in a single location (e.g., in a home environment, in an office environment, or as a server farm), while in other embodiments, the processors may be distributed across multiple locations.
[0143] The one or more processors may also be operable to support execution of the related operations in a "cloud computing" environment or as a SaaS. For example, as described above, at least some of the operations may be performed by a set of computers (as an example of a machine including a processor), which may be accessed via a network (e.g., the Internet) and one or more appropriate interfaces (e.g., APIs).
[0144] Furthermore, the figures depict some embodiments of the example environment for illustrative purposes only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein may be employed without departing from the principles described herein.
[0145] After reading this disclosure, those skilled in the art will appreciate additional alternative structural and functional designs for determining locations and routes through natural dialogue through the principles disclosed herein. Therefore, although specific embodiments and applications have been illustrated and described, it should be understood that the disclosed embodiments are not limited to the precise constructions and components disclosed herein. Various modifications, changes and variations may be made to the arrangement, operation and details of the methods and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims, and the modifications, changes and variations will be apparent to those skilled in the art.
Claims
1. A method in a computing device for determining a location and a route through natural conversation, the method comprising: receiving voice input including a search query from a user initiating a navigation session; generating, by one or more processors, a navigation search result set in response to the search query, the navigation search result set including a plurality of destinations or a plurality of routes corresponding to one or more destinations; providing, by the one or more processors, to the user an audio request for refining the set of navigation search results; In response to the audio request, receiving a subsequent voice input from the user comprising a refined search query; and One or more refined navigation search results including a subset of the plurality of destinations or the plurality of routes are provided, by the one or more processors, in response to the refined search query.
2. The method of claim 1, further comprising: Transcribing the speech input into a text set by an automatic speech recognition (ASR) engine; (a) providing, by the one or more processors, to the user the audio request for refining the set of navigation search results; (b) receiving, in response to the audio request, the subsequent voice input from the user including the refined search query; (c) filtering, by the one or more processors, the set of navigation search results by deleting a route from the plurality of routes based on the subsequent speech input to generate the one or more refined navigation search results; (d) determining, by the one or more processors, whether to provide a subsequent audio request to the user based on the one or more refined navigation search results; as well as (e) Iteratively executing (a) to (d) until the one or more refined navigation search results meet a threshold.
3. The method of claim 2, wherein filtering the set of navigation search results to generate the one or more refined navigation search results further comprises: The one or more processors executing a machine learning (ML) model are configured to remove routes from the set of routes having corresponding relevance scores that do not meet a relevance threshold based on a natural language transcription of the subsequent speech input, wherein the natural language transcription is not parsed, and the ML model is configured to receive the natural language transcription and the routes as inputs to output a relevance score for each route.
4. The method of claim 1, further comprising: The one or more refined navigation search results are provided at a user interface for viewing by the user.
5. The method of claim 1, further comprising: The one or more processors determine whether to provide the audio request to the user based on at least one of: (i) the total number of routes included in the multiple routes, (ii) the device type of the device used by the user to provide the voice input, (iii) the type of input provided by the user, or (iv) the number of second routes included in the multiple routes that meet the quality threshold.
6. The method of claim 1, further comprising: The audio request is transmitted verbally via a text-to-speech (TTS) engine for consideration by the user.
7. The method of claim 1, further comprising: receiving a verbal route acceptance input from the user, the verbal route acceptance input indicating an accepted route from the one or more refined navigation search results; displaying the accepted route at the user interface for viewing by the user; as well as The navigation session along the accepted route is initiated by the one or more processors by providing verbal navigation instructions corresponding to the accepted route as the user travels along the accepted route.
8. The method of claim 1, wherein generating the navigation search result set in response to the search query further comprises: transcribing the speech input into a text collection; as well as A machine learning (ML) model is applied by the one or more processors to the text collection to output user intent and a destination, wherein the ML model is trained using one or more text training data sets to output one or more training intents and one or more training destinations.
9. The method of claim 1, further comprising: transcribing the speech input into a text collection; parsing, by the one or more processors, the text collection to determine a destination value; extracting, by the one or more processors, the destination value from the text collection; as well as The destination value is searched, by the one or more processors, in a destination database.
10. The method of claim 9, further comprising: identifying, by the one or more processors, the plurality of destinations based on results of searching the destination database; as well as One or more routes to each of the plurality of destinations are generated by the one or more processors.
11. The method of claim 1 , wherein generating the navigation search result set in response to the search query further comprises: The one or more processors generate the one or more candidate routes to each of the multiple destinations based on a corresponding attribute set of each candidate route in the one or more candidate routes, wherein each corresponding attribute set includes one or more of the following: (i) transportation mode, (ii) number of transfers, (iii) total driving distance, (iv) total driving time, (v) total driving distance on each included road, or (vi) total driving time on each included road.
12. The method of claim 1, wherein providing the audio request to the user for refining the set of navigation search results further comprises: determining, by the one or more processors, a primary attribute of the plurality of routes that will result in minimizing the plurality of routes; as well as The audio request is generated, by the one or more processors, for the user based on the primary attribute.
13. The method of claim 1, wherein providing the audio request to the user for refining the set of navigation search results further comprises: The audio request is generated by the one or more processors executing a large language model (LLM) based on attributes of the plurality of routes.
14. The method of claim 1, wherein providing one or more refined navigation search results in response to the refined search query further comprises: generating, by the one or more processors executing a large language model (LLM), a text summary for each route in the subset of the plurality of routes; as well as The subset of the plurality of routes and each corresponding text summary are provided at the user interface for viewing by the user.
15. The method of claim 1, further comprising: receiving a selection of an accepted route from the user to initiate the navigation session along the accepted route; determining during the navigation session that an alternate route improves one of (i) a user's arrival time, (ii) a user's travel distance, or (iii) a user's time on a particular road; as well as The user is prompted with a verbal prompt or a text prompt during the navigation session with the option to switch from the selected route to the alternate route.
16. The method of claim 1, further comprising: The voice input and the subsequent voice input are recognized, by the one or more processors, based on a trigger phrase included as part of both the voice input and the subsequent voice input.
17. A computing device for determining a location and a route through natural conversation, the computing device comprising: user interface; one or more processors; as well as a computer-readable memory coupled to the one or more processors and storing thereon instructions that, when executed by the one or more processors, cause the computing device to: receiving a speech input including a search query from a user initiating a navigation session, generating a navigation search result set in response to the search query, the navigation search result set including a plurality of destinations or a plurality of routes corresponding to one or more destinations, providing an audio request to the user for refining the set of navigation search results, receiving, in response to the audio request, subsequent voice input from the user comprising a refined search query, and One or more refined navigation search results are provided in response to the refined search query that include a subset of the plurality of destinations or the plurality of routes.
18. The computing device of claim 17, wherein the instructions, when executed by the one or more processors, cause the computing device to: Transcribing the speech input into a text set by an automatic speech recognition (ASR) engine; (a) providing the user with the audio request for refining the set of navigation search results; (b) receiving, in response to the audio request, the subsequent voice input from the user including the refined search query; (c) filtering the set of navigation search results by deleting a route from the plurality of routes based on the subsequent speech input to generate the one or more refined navigation search results; (d) determining whether to provide a subsequent audio request to the user based on the one or more refined navigation search results; as well as (e) Iteratively executing (a) to (d) until the one or more refined navigation search results meet a threshold.
19. A computer-readable medium storing instructions for determining a location and a route through natural conversation, the instructions, when executed by one or more processors, causing the one or more processors to: receiving voice input including a search query from a user initiating a navigation session; generating a navigation search result set in response to the search query, the navigation search result set including a plurality of destinations or a plurality of routes corresponding to one or more destinations; providing an audio request to the user for refining the set of navigation search results; In response to the audio request, receiving a subsequent voice input from the user comprising a refined search query; and One or more refined navigation search results are provided in response to the refined search query that include a subset of the plurality of destinations or the plurality of routes.
20. The computer-readable storage medium of claim 19, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to: Transcribing the speech input into a text set by an automatic speech recognition (ASR) engine; (a) providing the user with the audio request for refining the set of navigation search results; (b) receiving, in response to the audio request, the subsequent voice input from the user including the refined search query; (c) filtering the set of navigation search results by deleting a route from the plurality of routes based on the subsequent speech input to generate the one or more refined navigation search results; (d) determining whether to provide a subsequent audio request to the user based on the one or more refined navigation search results; as well as (e) Iteratively executing (a) to (d) until the one or more refined navigation search results meet a threshold.
21. A method in a computing device for determining a location and a route through natural conversation, the method comprising: receiving input from a user to initiate a navigation session; generating, by one or more processors, one or more destinations or one or more routes in response to the user input; providing, by the one or more processors, a request to the user to refine a response to the user input; receiving subsequent input from the user in response to the request; as well as One or more updated destinations or one or more updated routes are provided by the one or more processors in response to the subsequent user input.
22. The method of claim 21, wherein the user input is a voice input or a text input, and the request is an audio request or a text request.
