Location-based mode of biasing the supply of content when the automated assistant is responding to a reduced natural language input

Abstract

Embodiments set forth herein relate to an automated assistant that operates according to various different location-based biasing modes in order to render responsive content for a user and / or proactively suggest content to a user. A user can provide a concise spoken utterance to the automated assistant while the automated assistant is operating according to one or more location-based biasing modes, but still receive accurate responsive output from the automated assistant. The responsive output is generated by biasing towards a subset of location characteristic data that has been prioritized over other subsets of location characteristic data. The biasing allows the automated assistant to compensate for any details that can be omitted from the spoken utterance, but allows the user to provide a shorter spoken utterance, thereby reducing the amount of language processing when processing input from the user.

Description

Background Technology

[0001] Humans can engage in human-computer dialogue with interactive software applications referred to herein as “automatic assistants” (also known as “digital agents,” “chatbots,” “interactive personal assistants,” “intelligent personal assistants,” “conversational agents,” etc.). For example, humans (who may be referred to as “users” when interacting with an automatic assistant) can provide commands and / or requests using spoken natural language input (i.e., utterances) that can in some cases be converted into text and then processed, and / or by providing natural language input in the form of text (e.g., typed text).

[0002] When a user is visiting a point of interest (POI), they can provide a number to the automated assistant to retrieve information and / or perform specific actions associated with that POI. However, given the limited battery power of most portable electronic devices, frequent and / or prolonged input to the automated assistant can consume energy and computing resources on the corresponding portable device. These limitations can be particularly problematic when users are in unfamiliar areas, cannot access a power source to recharge their portable devices, and / or cannot speak the local language, thus restricting their access to information when their portable devices are out of battery. Furthermore, when visiting unfamiliar areas, users may not recognize the most useful amenities until they have reached the end of their visit. As a result, users may unnecessarily consume resources due to their unfamiliarity with the availability of amenities such as train timetables, service counters, restroom locations, hospitals, charging stations, and / or other amenities that might be useful during their visit. Although users can inquire about certain amenities from their corresponding automated assistants, such frequent and / or prolonged inquiries can consume energy and other computing resources, thus accelerating the battery drain of their corresponding portable electronic devices. Summary of the Invention

[0003] The implementations described herein relate to operating an automated assistant for a client device based on various location-based bias patterns to respond to user input (e.g., spoken words) and / or pre-proactively render content. In some implementations, the location-based bias patterns available to the user's client device at a given location include at least a first location-based bias pattern and a second location-based bias pattern. The first location-based bias pattern biases the responsive rendering and / or pre-proactive rendering of content toward that location to a first limit. The second location-based bias pattern biases the responsive rendering and / or pre-proactive rendering of content toward that location to a second limit greater than the first limit. The biases in the first and / or second location-based bias patterns can include, for example, speech-to-text bias, content score bias, source bias, and / or query modification bias. For example, in the first location-based bias pattern, each of the preceding biases can reach a corresponding first limit, and in the second location-based bias pattern, each of the preceding biases can reach a corresponding second, larger limit. As another example, in the first location-based bias mode, speech-to-text bias can occur without query-modified bias, while in the second location-based bias mode, query-modified bias and speech-to-text bias (along with optionally more extensive speech-to-text bias) can occur.

[0004] As a result of the greater bias in the second location-based bias mode, users can provide more concise spoken utterances and / or other input via client devices (at least relative to the first location-based bias mode) to enable the automated assistant to provide location-specific information and / or perform location-specific actions in response. More concise input saves battery life and / or (multiple) other resources on the client device because the client device resources required to process the input are active for less time when processing more concise input. Furthermore, more concise input reduces the total duration of user / automated assistant interactions occurring via the client device, further saving battery life and / or (multiple) other client device resources. Moreover, in various implementations, the greater bias in the second location-based bias mode provides more efficient client and / or server parsing of spoken utterances and / or other input. As an example, and as described in more detail herein, in the second location-based bias mode, it is possible to significantly bias (or even limit) the corpus of content considered when responding to user input towards a subset of content, such as a subset of content with defined relationships to location. As another example, and as described in more detail herein, the second location-based bias mode is able to more aggressively rewrite queries based on location so that less content is considered to respond to the query.

[0005] While a second location-based bias mode offers various technical benefits when seeking location-specific content and / or actions, the embodiments disclosed herein can enter a second location-based bias mode in response to the satisfaction of certain conditions to avoid unintentionally over-biasing toward that location (and potentially negating the technical benefits). For example, a first location-based bias mode can be utilized when certain conditions are not met. As described herein, the first location-based bias mode provides a degree of bias toward that location, but this degree of bias is less than that of the second location-based bias mode. The conditions(s) for entering the second location-based bias mode can include, for example, positive user input (pre-initiated or in response to automatically generated prompts) indicating a desire to enter the second location-based bias mode. The conditions(s) can additionally or alternatively include being in that location and determining the presence of one or more contextual conditions. The contextual conditions(s) may include, for example, determining that the client device and / or user has never been to the location (or has been in the location less than a threshold number of times); determining that the user's(s) emails, search history, and / or calendar data indicate a specific purpose for being in the location; and / or other contextual conditions(s). In some implementations, determining that the client device is in the location and that the contextual conditions(s) exist enables the auto-assistant to automatically switch to a second location-based bias mode for the client device. In some other implementations, determining that the client device is in the location and that the contextual conditions(s) exist enables a prompt to be rendered via the client device, and the auto-assistant to automatically switch to a second location-based bias mode for the client device in response to receiving positive input from the prompt. In various implementations, the provided prompt and / or other rendered output notify the user of the client device that they have entered the second location-based bias mode, thereby providing an implicit queue for the user to provide more concise input.

[0006] Now, some non-limiting examples are provided of how bias can differ between a first location-based bias mode and a second location-based bias mode. For example, when operating according to the first location-based bias mode, speech-to-text processing can be biased for a specific portion of each spoken utterance (e.g., a first word and / or a second word) and / or biased toward the first word set. However, when operating according to the second location-based bias mode, speech-to-text processing can be biased relative to speech-to-text processing in the first location-based bias mode for a larger portion of each spoken utterance (e.g., all words or less than all words), and / or biased toward a second word set that is different from and / or includes more words than the first word set. Additionally or alternatively, when operating according to the first location-based bias mode, document and / or data source scoring can be biased to a certain extent for a particular location, while when operating according to the second location-based bias mode, document and / or data source scoring can be biased to a greater extent for a particular location. Alternatively or additionally, when operating in the first location-based bias mode, certain parts of speech (e.g., pronouns) of the query derived from spoken discourse can be modified to be replaced by region-related aliases (e.g., city names, region names, state names, village names, etc.). Furthermore, in the second location-based bias mode, other parts of speech of the query can be modified, and / or the query can be supplemented with location-specific aliases (e.g., names of locations of interest such as statues, artworks, historical landmarks, etc.). These modes can switch between responding to prompts, when a user enters a specific location, when the automated assistant determines that the user has followed suggestions rendered by the automated assistant, and / or responding to any other instructions that can be associated with changes in the user's context.

[0007] As an example, a user might be visiting Singapore as part of an international holiday and might be interested in visiting certain landmarks and trying certain foods in the area. Because Singapore could be an unfamiliar region to the user, they might access their automated assistant for information more frequently than they would in their home country. For instance, while visiting Singapore, a user might encounter landmarks they want to learn more about but may not know their names. Therefore, to ask certain questions about that particular landmark, the user could first provide multiple queries to determine the name of the landmark they would be referring to. For example, a user could invoke the automated assistant via their portable computing device by providing spoken phrases such as, “Assistant, is it possible to book a room in the large building in Singapore that looks like it has a boat on top of it?” Depending on the bandwidth of the network to which the portable computing device is connected and the versatility of voice-to-text processing available via the portable computing device, the automated assistant might not be able to provide an appropriate response within a reasonable timeframe. For example, because spoken utterances may be longer than the average spoken utterance from a user, portable computing devices can exhibit a certain amount of latency in order to both understand spoken utterances and provide responses to them.

[0008] To mitigate this latency and conserve computing resources such as power and processing bandwidth, automated assistants can operate according to one or more location-based bias patterns. These patterns enable the automated assistant to allow users to provide input that consumes substantially fewer computing resources when processed by a portable computing device. Alternatively or additionally, while the automated assistant is operating according to one or more location-based bias patterns, it can proactively render content for the user in advance to further eliminate the need for the user to engage in extensive conversational sessions to gather information that would otherwise be conveyed through pre-rendered content. Alternatively or additionally, while the automated assistant is operating according to one or more location-based bias patterns, query modifications can be performed based on the specific location-based bias pattern under which the automated assistant is operating. For example, certain location-based bias patterns enable the auto-assistant to operate based on a first location-based bias pattern (e.g., a location-specific bias pattern) or a second location-based bias pattern (e.g., a landmark-specific bias pattern), thereby allowing the auto-assistant to modify the input query (e.g., "Assistant, how tall is that [in Singapore]?") based on whether the user is in a general location (e.g., "Assistant, how tall is that [in Singapore]?") and / or near a more specific location (e.g., "Assistant, how tall is [The Sky Park]?").

[0009] To facilitate the aforementioned example, when users are navigating within their international access area, their portable computing devices can provide prompts to allow users to choose whether to enter a location-based bias mode. If the user chooses to enter location-based bias mode, the auto-assistant can be notified of certain locations of interest within the user's area. When the auto-assistant is notified that the user is near a specific location of interest, such as a landmark, it can operate to bias certain inputs and / or outputs based on data associated with the landmark. As an example, instead of the user providing a prior query (i.e., "Assistant, is it possible to book a room in the large building in Singapore that looks like it has a boat on top of it?"), the auto-assistant can proactively render content in advance via one or more interfaces of the portable computing device to inform the user of the name of a specific landmark. For example, the name of the specific landmark could be "Sky Park," and the auto-assistant could render the name "Sky Park" at the portable computing device's display interface before receiving relevant input from the user. In this way, if a user has the same query as before, the user can provide a shorter spoken utterance to convey the same inquiry (e.g., “Assistant, is it possible to book a room in The Sky Park?”). In this way, computing resources can be reserved when they are most critical, such as when users are in unfamiliar locations and therefore may need to reserve certain resources in emergencies or when they enter areas with limited network availability.

[0010] In some implementations, when identifying content to be provided to users located within a specific area and / or near specific locations of interest within that area, certain subsets of data can be rated higher than other subsets to identify that content. For example, to bias towards certain content while an auto-assistant is operating according to a specific location-based bias pattern, certain subsets of location-specific data can be rated to elevate certain content above others. As an example, when the auto-assistant is not operating according to any specific location-based bias pattern and the user provides spoken utterances, the auto-assistant can render response outputs based on a variety of different data. This different data can be associated with the user (e.g., calendar data, application data, message data, etc.), spoken utterances, a given context, and / or any other relevant information. However, when the user has visited a specific area and / or a specific location of interest, such as a landmark, the data used to render the response output can be processed according to a specific location-based bias pattern.

[0011] As an example, location feature data can be used to provide details about certain characteristics of an area a user is visiting, as well as one or more locations of interest within that area. When a user arrives at the area with their corresponding portable computing device, the automated assistant can switch to operating according to a first location-based bias pattern. As a result, the location feature data can be processed to identify a subset of the location feature data so that the bias is maximized compared to other location feature data that is not part of the subset, at least for the purpose of using the subset to render response content for the user. For example, following the previous example, when a user is located in Singapore and provides an inquiry about a certain area of ​​Singapore, a score can be assigned to a subset of location feature data corresponding to that area. This score can prioritize the subset of location feature data over other location feature data, at least for the purpose of generating response output for the user.

[0012] Furthermore, when a user is located within a region that includes a location of interest (e.g., "Sky Park"), and the user provides a query about that location of interest, a specific score can be assigned to a separate subset of the location feature data. This specific score allows that separate subset of the location feature data to take precedence over other location feature data and subsets of location feature data corresponding to a region that has been previously assigned a score. In this way, if a specific score of a separate subset of the location feature data takes precedence over any other assigned score, that separate subset of the location feature data can be used as a basis for suggesting content to the user and / or rendering response content. As an example, a subset of location feature data that takes precedence over all other location feature data can be used as a basis for modifying queries from the user to provide automatic suggestions to the user as the user moves toward a location of interest, and / or otherwise biased toward providing content that takes precedence over other content for rendering to the user when the user is near a location of interest within the region.

[0013] The above description is provided as an overview of some embodiments of this disclosure. Further descriptions of those and other embodiments are given below in more detail.

[0014] Other embodiments may include a non-transitory computer-readable storage medium storing instructions that can be executed by one or more processors (e.g., multiple central processing units (CPUs), multiple graphics processing units (GPUs), and / or multiple tensor processing units (TPUs)) to perform methods such as those described above and / or elsewhere herein. However, other embodiments may include a system of one or more computers including one or more processors operable to execute the stored instructions to perform methods such as those described above and / or elsewhere herein.

[0015] It should be understood that all combinations of the foregoing concepts and other concepts described in more detail herein are contemplated as part of the subject matter disclosed herein. For example, all combinations of the claimed concepts appearing at the end of this disclosure are contemplated as part of the subject matter disclosed herein. Attached Figure Description

[0016] Figure 1A , Figure 1B and Figure 1C The illustration shows a view of a user interacting with an automated assistant that operates according to one or more location-based bias patterns.

[0017] Figure 1D and Figure 1E The diagram illustrates a view of how the automated assistant processes input and / or output based on one or more location-based bias patterns when the user is in the same location.

[0018] Figure 2 The diagram illustrates a system for providing an automated assistant capable of biasing inputs and / or outputs according to one or more location-based bias patterns to conserve computing resources while a user is accessing the system.

[0019] Figure 3 The diagram illustrates a method for generating responsive content from an automated assistant capable of operating according to various location-based bias patterns to render content that can be used to mitigate the waste of computational resources.

[0020] Figure 4A and Figure 4B The diagram illustrates (multiple) methods for rendering response output for an auto-assistant based on one or more location-based bias patterns.

[0021] Figure 5 This is a block diagram of an example computer system. Detailed Implementation

[0022] Figure 1A , Figure 1B and Figure 1CViews 100, 120, and 150 illustrate user 102 interacting with an automated assistant operating according to one or more location-based bias patterns. Specifically, Figure 1A The illustration shows a view 100 of user 102 interacting with an automated assistant 108 accessible via a computing device 104 located within user 102's home. The automated assistant 108 is capable of receiving requests from user 102 to control one or more applications on computing device 104 and / or one or more other devices communicating with computing device 104. For example, user 102 can control television 110 to provide graphical content to the automated assistant 108 in response to a request. With prior permission from user 102, the automated assistant 108 can identify the content being rendered at television 110 to generate contextual data that can be used for processing to bias inputs and / or outputs.

[0023] In some embodiments of the automated assistant 108, first geographic characteristic data 106 can be generated to detail the characteristics of the user 102's location. The first geographic characteristic data 106 can also be used to determine whether the automated assistant 108 operates according to one or more location-based bias patterns. Location-based bias patterns can be activated based on geographic criteria established by the entity providing the automated assistant 108 and / or by a third-party entity different from the entity providing the automated assistant 108. For example, when the user 102 is located in a specific area outside their home, the third-party entity can establish geographic criteria for operating the automated assistant 108 in a first location-based bias pattern. Furthermore, when the user 102 is located outside their home and near a specific location of interest within a specific area, another third-party entity can establish geographic criteria for operating the automated assistant 108 in a second location-based bias pattern. However, when the first geographic characteristic data 106 does not satisfy any geographic criteria corresponding to the area of ​​interest or location, the automated assistant 108 can operate according to an operating pattern based on spoken utterances and / or other contextual data processed to provide a responsive output to the user 102.

[0024] For example, when user 102 is watching television 110 in their home, content related to a museum in another country can be rendered on television 110. To obtain more information about the content, user 102 can provide spoken words 114 to the automated assistant 108. These spoken words could be, for example, “How much is that?” In some implementations, the automated assistant 108, with prior permission from user 102, can use speech recognition to determine whether user 102 is the person interacting with the automated assistant 108. Additionally, computing device 104 can first generate geographic characteristic data 106 indicating that user 102 is interacting with the automated assistant 108 within their home. Furthermore, the automated assistant 108 can access context data characterizing the content being rendered on television 110. Based on user 102 being in their home, the automated assistant 108 can bypass location-based bias patterns and provide responsive output to user 102 based on context data and spoken words. For example, in response to spoken utterance 114, the automated assistant 108 can provide a response output 116 such as “The Thinker, by Auguste Rodin, is worth at least $15 million.”

[0025] In some implementations, the response output 116 can be stored as context data and thus used to generate subsequent response data in response to input from user 102. For example, user 102 can provide a subsequent spoken statement 114 such as “Are there science museums there?” In response, the automation assistant 108 can access context data identifying content that has been and / or is being rendered on television 110 and that is embodied through any response output provided by the automation assistant 108. Based on this context data, the automation assistant 108 can provide another response output 116, such as “The Discovery Museum and Curie Museum are located there.” This response output can be based on data associated with the content of the previous response output provided by the automation assistant 108 (e.g., “Thinker”) and / or search results generated by running a search command identifying the content of the subsequent spoken statement and the content of the previous response output.

[0026] In some implementations, the automatic assistant 108 can establish a unique biased operating mode as the user 102 acknowledges certain response outputs over time. This occurs when certain scenario data satisfies... Figure 1AThe operating mode can be activated when the scenario criteria corresponding to the scenario detailed in the description are met. When the automated assistant 108 is operating according to this unique bias operating mode, the input to and / or the output from the automated assistant 108 can be processed using a bias toward a subset of the source and / or data that was previously used to provide the response output as the basis for generating the unique bias operating mode (e.g., the website(s) that are the source of the content for the search results). In this way, various unique bias operating modes can be uniquely generated for different users interacting with the automated assistant 108. This allows the automated assistant 108 to more easily respond to concise input, such as when user 102 uses pronouns and / or other language substitutions for specific details and / or proper names to provide concise input.

[0027] Figure 1B The illustration shows a view 120 of user 102 when user 102 has left their respective home to access area 122 outside their residence. When user 102 is accessing an area outside their home, such as a location already suggested by automation assistant 108 or identified by content accessible to automation assistant 108, user 102 can be prompted at portable computing device 138 whether they want automation assistant 108 to switch to a first location-based bias mode. The prompt can be provided based on second geographic characteristic data 142 generated by computing device 140 (i.e., portable computing device 138) or otherwise accessible to it. The second geographic characteristic data 142, with prior permission from user 102, can optionally indicate that user 102 is at a specific location 130 adjacent to a public transfer station 132 within area 122.

[0028] When user 102 selects to have the auto assistant 108 operate according to a first location-based bias mode, the input to and / or output from the auto assistant 108 can be biased relative to the data used to process the input and / or output. Specifically, when operating in the first location-based bias mode, processing can be biased based on location characteristic data representing region 122 rather than other data that does not represent region 122. In some embodiments, the auto assistant 108 can automatically switch to the first location-based bias mode in response to user 102 relocating to region 122 previously identified by the auto assistant 108 and / or identified by content accessible to the auto assistant 108 (e.g., content requested by user 102 for video playback while user 102 is interacting with the auto assistant 108). For example, because user 102 provides a rendering that results in output identifying a specific landmark located in region 122 (e.g., "The Thinker"), the auto assistant 108 can automatically switch to the first location-based bias mode in response to user 102 arriving at region 122. In this way, the automated assistant 108 can operate to guide user 102 through area 122 based on the assumption that user 102 is interested in learning more about area 122. In some implementations, the bias of processing input and / or output can be based on whether the data being processed is associated with or otherwise identifies content previously rendered during the interaction between user 102 and automated assistant 108. For example, when operating in a first location-based bias mode while user 102 is in area 122, computing device 140 can rank certain documents and / or data sources identifying "thinkers" to a higher degree than documents and / or data sources not identifying "thinkers". Furthermore, when operating in a second location-based bias mode while user 102 is in area 122, computing device 140 can rank specific documents and / or data sources identifying "thinkers" to a greater degree than they would be ranked in the first location-based bias mode.

[0029] As an example, when user 102 is at location 130 near public transit station 132, user 102 can provide a spoken utterance 134 such as “How much is this?”. In response, the automated assistant 108 can access content used to generate a response output. However, when processing the content, the automated assistant 108 can be biased towards location characteristic data representing region 122 and location 130. Specifically, computing device 140 can score one or more subsets of data provided by one or more sources based on their relevance to the spoken utterance 134, and then modify one or more assigned scores to prioritize certain subsets of data over others. The subsets of data that are prioritized over others can correspond to sources representing location characteristic data of public transit station 132. Thus, when responding to the question “How much is this?”, the automated assistant 108 can generate a response output based on the most prioritized subset of data (e.g., fare schedules provided by a public subway station website). For example, the automatic assistant 108 enables the computing device 140 to render a response output 136 such as "The Metro is $20 for 3 days of unlimited riding".

[0030] In some implementations, computing device 140 can perform query rewriting to bias the input and / or output of automation assistant 108 when operating in a first location-based bias mode. Specifically, computing device 140 can receive spoken utterance 134 from user 102, convert audio data generated from spoken utterance 134 into natural language content, and then modify the natural language content according to the first location-based bias mode before running a query based on the natural language content. For example, automation assistant 108 can bias the query rewriting based on the highest priority subset of location feature data, thereby using the word "The Metro" to evoke the word "this" from the natural language content. Thus, the natural language content incorporated into the query to generate a suitable response output could be: "How much is [The Metro]?" As a result of running this query, automation assistant 108 can identify information about public transfer station 132 and incorporate this data into the response output 136.

[0031] In some implementations, while the automated assistant 108 is operating according to a first location-based bias pattern 128, the user 102 can provide spoken utterances 134 related to region 122 rather than a specific location 130 where the user 102 is currently located (e.g., a public transit station 132). For example, when the user 102 provides other spoken utterances 144 such as “Where can I learn about science?”, the computing device 140 can generate and / or access second geographic feature data 142 and generate feature embeddings from that second geographic feature data. Additionally, the computing device 140 can generate and / or access result data corresponding to the results of a query executed based on natural language content from the other spoken utterances 144. When the computing device 140 identifies certain results that satisfy the query to a particular extent, the computing device 140 can generate embeddings from these results to determine the distance of each embedding to the feature embedding based on the second geographic feature data 142. Each embedding can then be scored based on its distance from the feature embedding, and one or more embeddings corresponding to the shortest distance can be used as the basis for providing another response output 146. For example, the first nearest embedding can correspond to the science museum located in area 122 at a specific location of interest 124. Additionally, the second nearest embedding can correspond to a subway station near one or more science museums and within a threshold distance 126 from the location of interest 124. The first and second nearest embeddings can be used as the basis for providing another response output 146 such as "The Curie Museum is near the Monge Metro stop, named after Gaspard Monge".

[0032] If user 102 selects to visit a destination suggested by the auto assistant 108 while the auto assistant 108 is operating in a first location-based bias mode, the auto assistant 108 can determine when user 102 has arrived at the suggested destination and, in response, prompt user 102 regarding operation in a second location-based bias mode. In other words, because the auto assistant 108 suggested a specific destination while operating in the first location-based bias mode, and user 102 selected to go to that destination, the auto assistant 108 can prompt user 102 whether they wish the auto assistant 108 to operate according to the second location-based bias mode.

[0033] Figure 1CThe illustration shows a view 150 of user 102 interacting with the automation assistant 108 while it is operating according to a second location-based bias mode 160. User 102 can choose to operate the automation assistant 108 according to the second location-based bias mode by providing input to computing device 140 in response to prompts from the automation assistant 108 regarding the second location-based bias mode 160. When operating in the second location-based bias mode, the automation assistant 108 can bias the processing of inputs to and / or outputs from the automation assistant 108 based on the user 102's location of interest 124. For example, user 102 can reposition to a specific location 130 within a threshold distance 126 from the location of interest 124 and within a specific region 122. When user 102 provides a query to the automation assistant 108, the processing of the input can be biased according to the second location-based bias mode, in which data corresponding to the location of interest 124 takes precedence over other data. Optionally, data corresponding to location 124 of interest can be prioritized over other data corresponding to region 122.

[0034] For example, user 102 can provide spoken utterance 156 such as “How much is this?”. In response, computing device 140 can generate and / or access third geographic feature data 154 based on data representing location 124 and / or region 122 of interest from one or more sources. Computing device 140 and / or a separate computing device (e.g., a server device) can process the third geographic feature data 154 to generate feature embeddings from the third geographic feature data 154. Furthermore, computing device 140 and / or a separate computing device (e.g., a server device) can process audio data from the spoken utterance 156 and run queries to identify data suitable for use when responding to the spoken utterance 156. One or more embeddings can then be generated from the data identified by running queries.

[0035] To bias the response output toward the generated feature embeddings, computing device 140 can identify a subset of embeddings having a minimum distance from the content corresponding to the spoken utterance 156. A score based on that distance can then be assigned to each subset of embeddings. Furthermore, computing device 140 can then identify specific subsets within the subset of embeddings having a minimum distance from the feature embeddings. Each score assigned to those specific subsets can be modified so that certain subsets having a minimum distance from the feature embeddings in the embedding space take precedence over other subsets of embeddings. The embedding with the highest or most prioritized score can then be identified, and a subset of the data based on the embeddings can be used as the basis for generating a response output 158 ​​in response to the spoken utterance 156. For example, the highest-priority subset of data could correspond to a museum website providing a daily visit schedule for a location of interest (e.g., the Curie Museum). The subset of data can be used to generate a response output 158 ​​such as "The Curie Museum is free but encourages donations."

[0036] In some implementations, while the automated assistant 108 remains in a second location-based bias mode as the user 102 continues to operate it, the automated assistant 108 is able to continue biasing the processing of inputs directed to it and / or outputs from it towards a highest priority subset of data (e.g., data from a museum website). In this way, as the user 102 remains within a threshold distance 126 of the location of interest 124, they will receive a response output from the automated assistant 108 based on the highest priority subset of data. For example, if the user 102 decides to enter the museum 152 at the location of interest 124 and provides the automated assistant 108 with a follow-up query about the exhibits within the museum (e.g., “How old is this laboratory?”), the automated assistant 108 is able to bias its response based on the highest priority subset of data corresponding to the location of interest. This conserves computing resources that might otherwise be consumed when processing longer queries from user 102 (e.g., a longer version of the previous query might be: "How old is the laboratory exhibit in the Curie Museum in France?").

[0037] In some implementations, the bias in processing input to the auto-assistant 108 when operating in a second location-based bias mode can include editing words in a query generated based on spoken utterance 156 from user 102. For example, when the auto-assistant 108 is operating according to the second location-based bias mode, the auto-assistant 108 can replace ambiguous words and / or pronouns with more specific words identified in the highest priority subset of the data. As an example, when the user provides the spoken utterance 156 “How much is this?”, the computing device 140 can replace the word “this” with another word or phrase from the highest priority subset of the data. The other word or phrase could be, for example, “a ticket to the Curie Museum”. Therefore, a query that can be run when operating in the second location-based bias mode could be “How much is [a ticket to the Curie Museum]?”

[0038] In this way, because user 102 has already accepted some tour suggestions from the automated assistant 108, the automated assistant 108 is able to operate to provide tours of certain areas including one or more locations of interest. Therefore, as long as user 102 is accepting some suggestions, the automated assistant 108 is able to continuously switch between a first location-based bias mode and a second location-based bias mode. This, at least in terms of responding to queries about a region (e.g., region 122) to queries about a specific location of interest (e.g., location 124), thus enables the automated assistant 108 to operate between subsets of data of different sizes. This adaptability of the automated assistant 108 allows it to provide more accurate responses to more concise inputs, such as spoken utterances. As a result, the computational resources of various computing devices can be preserved while users are visiting and therefore may be unfamiliar with the locations where their devices are charged and / or otherwise gathering more specific details to be submitted with their queries.

[0039] Figure 1D and Figure 1EViews 170 and 180 respectively illustrate how the automated assistant 108 processes input and / or output according to one or more location-based bias modes while the user 102 is in the same position, regardless of mode transitions. For example, when operating their portable computing device 138, the user 102 can provide instructions on what they expect from the portable computing device 138 and / or the automated assistant 108, based on a location-based bias mode. The user 102 can provide this instruction by explicitly inputting to the automated assistant 108, responding to prompts provided by the automated assistant 108, and / or otherwise operating the computing device 140 to indicate the expectation of operation of the computing device 140 according to a location-based bias mode.

[0040] While the automated assistant 108 is operating under state 172, where a first location-based bias pattern is active, the user 102 can be located at a specific location of interest 124, such as museum 152. When the user 102 is at location 124, the automated assistant 108 can bias the processing of input and / or output based on the user 102's location and the first location-based bias pattern. For example, the user 102 can provide a spoken utterance 174 such as "Open?" to retrieve information about nearby businesses that are open. In response, the automated assistant 108 can bypass the processing of the biased spoken utterance 174 for certain data exclusively associated with museum 152. Instead, the automated assistant 108 can bias processing based on a larger corpus of documents and / or data, such as application data accessible via computing device 140 (e.g., calendar data, messaging data, area-related location data, etc.). As a result, the automated assistant 108 is able to provide a response output 176 that may be related to area 122 rather than exclusively to location of interest 124 (e.g., museum 152). The response output 176 may be, for example, “Here arealist of openlocations in the area…[Names of Open Businesses]”.

[0041] In some implementations, user 102 can choose to switch the automatic assistant 108 to a second location-based bias mode, such as Figure 1EThe state 182 of the automated assistant 108 is changed accordingly, as provided in view 180. Furthermore, when the user 102 remains at the same specific location 130 while switching between modes, the processing of inputs and / or outputs from the automated assistant 108 can be modified accordingly. For example, if the user 102 indicates an expectation for the computing device 140 to operate according to a second location-based bias mode, the processing of certain data can be biased based on the location of interest 124. As a result, if the user 102 provides a spoken utterance 184 such as “Open?” to determine whether a specific location of interest 124 is open, one or more documents and / or data sources associated with the specific location of interest 124 can be biased to a greater extent than with other documents and / or data sources.

[0042] Alternatively, the natural language content of the query based on spoken utterance 184 can be edited to bias the query toward a specific location of interest 124. For example, the query can be modified to state "[Curie Museum] open?". Alternatively, the results from the running query can be re-ranked according to the degree to which each result is associated with the specific location of interest 124 (e.g., museum 152). As a result, when user 102 selects to enter a second location-based bias mode, computing device 140 can render content biased toward the specific location of interest 124. This allows user 102 to provide more concise spoken input without compromising the quality and / or accuracy of any response output, while also conserving computing resources.

[0043] In some implementations, the automated assistant 108 can switch to a second location-based bias mode in response to the user 102 being located at the location of interest 124 and at the automated assistant 108. Furthermore, the user 102 can bypass providing spoken utterances to the automated assistant 108 but still receive content according to the second location-based bias mode. For example, in response to the automated assistant 108 determining that the user 102 is located at the location of interest 124, the automated assistant 108 can proactively render web pages, images, documents, sounds, videos, and / or any other content that may be relevant to the person located at the location of interest 124. In some implementations, when the automated assistant 108 operates to proactively provide content in advance without initial input from the user 102, those suggestions can be ranked and / or prioritized based on one or more words corresponding to the location of interest.

[0044] As an example, when the automation assistant 108 determines that the user 102 is at a location of interest, the automation assistant can proactively render a timeline of the time when the user can access the features of the location of interest 124. This timeline can be rendered even if the user 102 has not previously requested it. Alternatively or additionally, content can be rendered in response to the user 102 accessing their device and / or otherwise providing some indication that they intend to interact with their computing device. For example, the user 102 can access a browser from their computing device to look up news about the area they are visiting, and in response to opening the browser, the automation assistant 108 can proactively provide information about the location of interest 124 in advance according to a second location-based bias pattern.

[0045] Figure 2 The illustration depicts a system 200 providing an automated assistant 204 capable of biasing inputs and / or outputs according to one or more location-based bias patterns to conserve computing resources while a user is accessing the system. The automated assistant 204 can operate as part of an assistant application provided at one or more computing devices, such as computing device 202, and / or server devices. A user can interact with the automated assistant 204 via assistant interfaces 220, which can be a microphone, multiple sensors (e.g., a GPS device), a camera, a touchscreen display, a user interface, and / or any other device capable of providing an interface between the user and the application. For example, a user can initialize the automated assistant 204 to perform functions (e.g., providing data, controlling peripherals, controlling individual applications, accessing agents, generating inputs and / or outputs, etc.) by providing verbal, textual, and / or graphical input to the assistant interface 220. The computing device 202 can include a display device, which can be a display panel including a touch interface for receiving touch input and / or gestures to allow a user to control an application 234 on the computing device 202 via the touch interface. In some embodiments, computing device 202 may lack a display device, thereby providing audible user interface output instead of graphical user interface output. Furthermore, computing device 202 may provide a user interface for receiving spoken natural language input from a user, such as a microphone. In some embodiments, computing device 202 may include a touch interface and may be without a camera, but may optionally include one or more other sensors.

[0046] Computing device 202 and / or other third-party client devices can communicate with the server device via a network such as the Internet. Additionally, computing device 202 and any other computing devices can communicate with each other via a local area network (LAN) such as a Wi-Fi network. Computing device 202 can offload computing tasks to the server device to preserve computing resources at computing device 202. For example, the server device can host the automated assistant 204, and / or computing device 202 can send input received at one or more assistant interfaces 220 to the server device. However, in some embodiments, the automated assistant 204 can be hosted at computing device 202 and can execute various processes associated with the automated assistant's operation at computing device 202.

[0047] In various implementations, all or less all aspects of the automated assistant 204 can be implemented on the computing device 202. In some of those implementations, aspects of the automated assistant 204 are implemented via the computing device 202 and are able to interface with a server device that is capable of implementing other aspects of the automated assistant 204. The server device may optionally serve multiple users and their associated assistant applications via multiple threads. In implementations where all or less all aspects of the automated assistant 204 are implemented via the computing device 202, the automated assistant 204 may be an application separate from the operating system of the computing device 202 (e.g., installed "on top of" the operating system) — or may alternatively be implemented directly by the operating system of the computing device 202 (e.g., considered an application of the operating system, but integrated with the operating system).

[0048] In some implementations, the automated assistant 204 may include an input processing engine 206 that may employ multiple different modules to process inputs and / or outputs from the computing device 202 and / or the server device. For example, the input processing engine 206 may include a voice processing engine 208 that can process audio data received at the assistant interface 220 to identify text embodied in the audio data. The audio data can be sent from, for example, the computing device 202 to the server device to conserve computing resources at the computing device 202. Alternatively, the audio data can be processed exclusively at the computing device 202.

[0049] The process for converting audio data into text can include a speech recognition algorithm that employs neural networks and / or statistical models to identify groups of audio data corresponding to words or phrases. The text converted from the audio data can be parsed by data parsing engine 210 and provided to the automation assistant 204 as a means to generate and / or identify command phrases, intents, actions, slot values, and / or any other content specified by the user. In some implementations, the output data provided by data parsing engine 210 can be provided to parameter engine 212 to determine whether the user has provided input related to a specific intent, action, and / or routine that can be performed by the automation assistant 204 and / or by an application or agent accessible via the automation assistant 204. For example, assistant data 238 can be stored at a server device and / or computing device 202 and can include data defining one or more actions that can be performed by the automation assistant 204, as well as parameters necessary to perform these actions. Parameter engine 212 can generate one or more parameters for intents, actions, and / or slot values ​​and provide these parameters to output generation engine 214. The output generation engine 214 can use one or more parameters to communicate with the assistant interface 220 to provide output to the user, and / or to communicate with one or more applications 234 to provide output to one or more applications 234.

[0050] In some implementations, the automated assistant 204 can be an application that can be installed "on top of" the operating system of the computing device 202 and / or it can itself form part (or all) of the operating system of the computing device 202. The automated assistant application includes and / or has access to on-device speech recognition, on-device natural language understanding, and on-device performance. For example, on-device speech recognition can be performed using an on-device speech recognition module that processes audio data (detected by microphone(s)) using an end-to-end speech recognition machine learning model stored locally at the computing device 202. The on-device speech recognition model can be used to generate recognized text for spoken utterances (if any) presenting the audio data. Alternatively, for example, on-device natural language understanding (NLU) can be performed using an on-device NLU module that processes the recognized text generated using on-device speech recognition and optionally uses context data to generate NLU data. The NLU data can include intent(s) corresponding to the spoken utterances and optionally parameters(s) for the intent(s)(s).

[0051] On-device execution can be performed using an on-device execution module that utilizes NLU data (from on-device NLU) and optionally other local data to determine multiple actions to be taken for parsing spoken utterance(s) and optionally multiple parameters for those intentions. This can include determining local and / or remote responses to the spoken utterance (e.g., answers), multiple interactions to be performed with multiple locally installed applications based on the spoken utterance, multiple commands to be sent (directly or via corresponding remote systems) to multiple Internet of Things (IoT) devices based on the spoken utterance, and / or multiple other parsing actions to be performed based on the spoken utterance. On-device execution can then initiate local and / or remote execution / running of the determined multiple actions to parse the spoken utterance.

[0052] In various implementations, remote speech processing, remote NLU, and / or remote execution can be utilized at least selectively. For example, recognized text can be selectively sent to at least a plurality of remote automation assistant components for remote NLU and / or remote execution. For example, recognized text can be optionally sent for remote execution in parallel with on-device execution or in response to failure of on-device NLU and / or on-device execution. However, on-device speech processing, on-device NLU, on-device execution, and / or on-device execution can be prioritized at least due to the reduced latency they provide when parsing spoken utterances (due to the elimination of client-server round trips to parse spoken utterances). Furthermore, on-device functionality can be the only functionality available in situations where there is no network connectivity or limited network connectivity.

[0053] In some implementations, computing device 202 may include one or more applications 234, which may be provided by a third-party entity different from the entity providing computing device 202 and / or automation assistant 204. The application state engine of automation assistant 204 and / or computing device 202 may access application data 230 to determine one or more actions that can be performed by the one or more applications 234 and the state of each of the one or more applications 234. Furthermore, application data 230 and / or any other data (e.g., device data 232) may be accessed by automation assistant 204 to generate context data 236, which may characterize the context in which a particular application 234 is operating on computing device 202 and / or the context in which a particular user is accessing computing device 202.

[0054] While one or more applications 234 are running at computing device 202, device data 232 can characterize the current operating state of each application 234 running at computing device 202. Furthermore, application data 230 can characterize one or more features of the running application 234, such as the content of one or more graphical user interfaces being rendered at the instruction of one or more applications 234. Alternatively or additionally, application data 230 can characterize action patterns that can be updated by the respective application and / or by the automation assistant 204 based on the current operating state of the respective application. Alternatively or additionally, one or more action patterns for one or more applications 234 can remain static but can be accessed by the application state engine to determine the appropriate action to be initialized via the automation assistant 204.

[0055] The computing device 202 may also include a geographic feature engine 216, which is capable of generating data characterizing one or more geographic features of a scene and / or environment. For example, when a user is operating their respective computing device 202, the geographic feature engine 216 can generate geographic feature data based on signals from one or more sensors communicating with the computing device 202. The geographic feature engine 216 can generate data characterizing the user's location relative to a specific area, such as a city, town, and / or any other geographic region. Furthermore, the geographic feature engine 216 can generate data related to locations of interest within the area where the user is located or corresponding to the user's location. Additionally, with the user's prior permission, the geographic feature engine 216 can generate data indicating whether the user is navigating towards or away from their residential area, such as their hometown city, town, and / or country.

[0056] In some implementations, computing device 202 may include a geographic criteria engine 218 for determining whether data generated by geographic feature engine 216 satisfies one or more geographic criteria. Geographic criteria may correspond to one or more location-based bias patterns defined by device data 232 and / or assistant data 238. For example, a location-based bias pattern may be a pattern in which inputs to and / or outputs from the automated assistant 204 are processed such that data relevant to a certain area is prioritized over other data. Alternatively, another location-based bias pattern may be a pattern in which inputs to and / or outputs from the automated assistant 204 are processed such that certain data characterizing a location of interest is prioritized over other data.

[0057] When the geographic criteria engine 218 determines that a user's geographic characteristics satisfy a specific geographic criterion corresponding to a specific location-based bias pattern, the geographic criteria engine 218 can identify geographic criteria that satisfy the pattern selection engine 222. In other words, when a first geographic criterion corresponding to a first location-based bias pattern has been satisfied, the geographic criteria engine 218 can indicate this to the pattern selection engine 222. Furthermore, when a second geographic criterion corresponding to a second location-based bias pattern has been satisfied, the geographic criteria engine 218 can indicate this to the pattern selection engine 222. Additionally, when no geographic criterion corresponding to any location-based bias pattern has been satisfied, the pattern selection engine 222 can indicate this to the pattern selection engine 222. The pattern selection engine 222 can either indicate this to the pattern selection engine 222 or bypass providing this indication to the pattern selection engine 222.

[0058] When the pattern selection engine 222 selects a location-based bias pattern according to already satisfied geographic criteria, it can instruct the location-based bias engine 224 to make that selection. The location-based bias engine 224 can use the selection of a specific location-based bias pattern to bias the processing of certain inputs and / or outputs according to the selected specific location-based bias pattern. For example, when a first location-based bias pattern is selected, the location-based bias engine 224 can cause the auto-assistant 204 to process certain inputs related to a specific region to be modified to include certain priority data. For example, priority data can be a subset of data from one or more sources related to inputs from the user and / or outputs from the auto-assistant, and provides details about certain characteristics of the geographic region. Furthermore, when a second location-based bias pattern is selected, the location-based bias engine 224 can cause the auto-assistant 204 to process certain inputs related to a specific region of interest to be modified to include certain other priority data. For example, other preferred data could be a separate subset of data from one or more other sources related to user input and / or output from an automated assistant, and provide details about certain features of the region of interest.

[0059] According to some implementations, speech-to-text processing based on a first location-based bias mode or a second location-based bias mode can generate multiple candidate text transcriptions and / or one or more candidate response transcriptions from a user's spoken utterance. Each candidate transcription can be associated with a corresponding score indicating the confidence level of the transcription's correctness. Such scores can be based on, for example, the degree of matching between phonemes and candidate transcriptions. When the automated assistant is operating according to the first location-based bias mode, the score of transcriptions including one or more words associated with the user's location can be biased and / or prioritized over other scores of transcriptions that do not include one or more words associated with that location. When the automated assistant is operating according to the second location-based bias mode, the score of transcriptions including one or more words associated with the user's location of interest can be biased and / or prioritized over other scores of transcriptions that do not include one or more words associated with the location of interest.

[0060] As an example, when a user is at home and provides a spoken utterance such as “Assistant, where is the Louvre?”, the automated assistant is able to generate multiple different candidate response transcriptions. When the user is at home, candidate transcriptions including words from sources associated with their home (e.g., a travel website provided by a US company) can be scored similarly to sources from other locations (e.g., France). However, when the user provides this spoken utterance while in Paris but not within a threshold distance of the Louvre, the automated assistant can operate according to a first location-based bias pattern. As a result, transcriptions including words from sources associated with France (e.g., French blogs about the Louvre, websites for Paris public transport) can be scored and / or prioritized over other transcriptions that do not include such words. Alternatively or additionally, when the user provides this spoken utterance while within a threshold distance of the Louvre, the automated assistant can operate according to a second location-based bias pattern. As a result, transcriptions containing words from sources associated with the Louvre (e.g., websites owned by the Louvre Museum) were able to be scored and / or prioritized over other transcriptions that did not include such words.

[0061] Figure 3The illustration illustrates a method 300 for generating responsive content from an automated assistant capable of operating according to various location-based bias patterns to allow users to provide concise spoken input when accessing various locations. Method 300 can be performed by one or more applications, computing devices, and / or any other means or modules capable of providing access to the automated assistant. Method 300 can include an operation 302 determining whether the automated assistant has received spoken utterances from one or more assistant interfaces of a portable computing device. When no spoken utterances have been received at the portable computing device, operation 302 can be repeated until one or more users provide spoken utterances to the automated assistant. However, when the automated assistant receives spoken utterances, method 300 can proceed from operation 302 to operation 304. It should be noted that it is possible to... Figure 3 The operations provided in the code are executed in a different order than those in method 300.

[0062] Operation 304 can include determining whether the user is located within an area that includes a location of interest. The area can be, for example, a geographical region, and the location of interest can be a landmark characterized by any public data accessible to a portable computing device. For example, a landmark can be a historical monument described by one or more public web pages and / or one or more applications accessible via a portable computing device. Thus, the user can use a browser application to navigate to a specific web page that includes information about the landmark. When it is determined that the user is not located within an area that includes a location of interest, method 300 can proceed from operation 304 to operation 314. Operation 314 can include generating a response output based on neither a first location-based bias pattern nor a second location-based bias pattern. For example, when the user is in their home, which may not include a landmark described by any public data, the automated assistant can generate a response output based on spoken words and / or any other data accessible to the automated assistant, without biasing to the extent that would otherwise occur under either the first or second location-based bias pattern. As an example, the spoken utterance could be, “Assistant, where can I go to learn more about Lincoln?”, and the response output generated by the automated assistant based on action 304 could be, “You can see a movie about former US President Lincoln at the movie theater 1.2 miles away.” In other words, according to the implementation discussed herein, the automated assistant is able to provide web results that are not specifically biased based on any location-based bias patterns used to bias the input to the automated assistant and / or the output from the automated assistant.

[0063] When it is determined in operation 304 that the user is located within an area including the location of interest, method 300 can proceed from operation 304 to operation 306. Operation 306 can include determining that the automated assistant is operating according to a first location-based bias pattern or a second location-based bias pattern. In some embodiments, operation 306 can be an operation that determines whether the user's context meets specific criteria associated with the first location-based bias pattern or the second location-based bias pattern. For example, when it is determined that the user is within a threshold distance of a specific area, method 300 can proceed from operation 306 to operation 310. However, when it is determined that the user is within another threshold distance of a location of interest within a specific area, method 300 can proceed from operation 306 to operation 308. Alternatively or additionally, the automated assistant can operate according to the first location-based bias pattern or the second location-based bias pattern depending on how the user responds to a specific prompt that requests the user to select from one or more location-based bias patterns.

[0064] Operation 310 can include generating a response output based on a first bias degree. The first bias degree can refer to a first limit to which the processing of the input and / or output is biased towards a specific subset of location characteristic data. For example, and following the previous example, processing of spoken utterances provided by the user can be performed based on a first limit of bias towards location characteristic data associated with the user's location. For example, when a user provides the spoken utterance “Assistant, where can I go to learn more about Lincoln?”, the user could be located in the District of Columbia. According to operation 310, the natural language content of the spoken utterance can be modified to bias the spoken utterance to further identify characteristics of the area where the user is located.

[0065] For example, a request embodied in spoken utterance can be modified to include an alias for the user's location, such as the alias "DC". As a result, the request can be modified to: "Where can I go to learn more about Lincoln [in DC]?" In this way, processing the modified request allows the automated assistant to generate a response output biased towards the user's location. Furthermore, this mitigates the waste of computational resources that might otherwise be consumed when requiring the user to specify an alias via spoken utterance, thereby enabling the processing of additional audio data on portable computing devices and / or any other relevant devices.

[0066] Operation 308 can include generating a response output based on a second bias, which can refer to a second limit to which the processing of inputs and / or outputs is biased towards a specific subset of location characteristic data. In some embodiments, the second limit to which the processing of inputs and / or outputs is biased can be greater than a first limit to which the inputs and / or outputs are biased. For example, when a user is within a threshold distance of the Lincoln Memorial in the District of Columbia, and / or the user's geographic characteristics meet geographic criteria associated with the Lincoln Memorial, the spoken utterance and / or response output can be biased based on the second bias. For example, in response to receiving the spoken utterance "Assistant, where can I go to learn more about Lincoln," and the user is located at the Lincoln Memorial, the processing of a subset of location characteristic data associated with the District of Columbia and the Lincoln Memorial can be biased such that this subset takes precedence over other subsets of location characteristic data associated with the District of Columbia.

[0067] In some implementations, the correlation between certain subsets of the location-specific data can be specified by one or more entities, such as a third-party entity different from the entity providing the automation assistant, for a specific location of interest within the area. Therefore, in response to received spoken utterances and based on location-based bias patterns and / or the user having geographical characteristics that satisfy geographical criteria associated with the Lincoln Memorial, a subset of data designated for the Lincoln Memorial can be identified by the automation assistant. This subset of data can be scored such that the score assigned to that subset can take precedence over other subsets of the location-specific data. In this way, when the automation assistant is generating a response output in response to spoken utterances, the subset of data designated for the Lincoln Memorial can be biased over other subsets of the data. As a result, the automation assistant can generate a response output such as “An entrance to a museum is located at the base of the Lincoln Memorial.” In some implementations, the subset of data can be biased based on the user’s trajectory when and / or after the user assumes they are speaking. For example, when a user's geographic characteristics meet geographic criteria corresponding to a location of interest, the user's trajectory can be used in conjunction with the geographic characteristics to bias the processing of a specific subset of the location characteristic data. For instance, based on the user's trajectory, a specific direction for reaching the museum located at the base of the Lincoln Memorial can be incorporated into the response output (e.g., "An entrance to amuseum is located at the base of the Lincoln Memorial and to the left as you approach the Lincoln Memorial.").

[0068] Method 300 may further include operation 312 of rendering the generated response output. Rendering the generated response output may include enabling the automated assistant to provide the response output via one or more interfaces of the portable computing device and / or any other computing device communicating with the portable computing device. For example, rendering the generated response output may include providing audio output embodying natural language content based on the response output generated by the automated assistant. Alternatively or additionally, rendering the generated response output may include providing graphical data embodying the response output generated by the automated assistant. In this way, access to useful information can be rationalized so that certain computing resources can be retained on the portable computing device and / or any other relevant device when the user is visiting away from their residence.

[0069] For example, output provided according to the second bias level can save power that might otherwise be consumed when displaying graphical content on the portable computing device's display interface, as users are spared the need to open a web browser to search for additional information about locations of interest. Furthermore, since many web browsers tend to consume processing bandwidth on computing devices, users can benefit from bypassing the use of a web browser by invoking an automated assistant to obtain information while accessing the site. In some implementations, location characteristic data can be preloaded in response to determining that a user is visiting a specific area including a particular location of interest away from their residence. This preloaded location characteristic data can be stored on the portable computing device and will therefore be accessible without being connected to a local area network and / or wide area network. Furthermore, in some implementations, the automated assistant can operate exclusively on the portable computing device, further conserving network resources that might be consumed when sending input and output between web servers. A bias can be applied to process a subset of the location characteristic data on the portable computing device and / or a separate server device. This bias can be applied in real time as the user moves between locations of interest within the area, arrives at the area, leaves the area, and / or otherwise displays dynamic geographic characteristics.

[0070] Figure 4A and Figure 4B The illustrations depict methods 400 and 420 for rendering response output for an automated assistant based on one or more location-based bias patterns. Methods 400 and 420 can be performed by one or more applications, computing devices, and / or any other means or modules capable of providing access to the automated assistant. Method 400 may include operation 402 determining whether spoken utterances are received by the computing device providing access to the automated assistant. The automated assistant is operable according to one or more operating modes including one or more location-based bias patterns. Each location-based bias pattern allows the automated assistant to bias input and / or output processing based on whether the user is accessing a region including a location of interest. When spoken utterances are received at the computing device, method 400 can proceed from operation 402 to operation 404. However, when no spoken utterances are detected at the computing device, the computing device can continue to monitor for spoken utterances directed to the automated assistant.

[0071] Operation 404 can include determining whether the user is located within an area including a location of interest. This area can be any geographic location, and the location of interest can be any specific location characterized by data available from one or more sources accessible via a computing device. For example, the area could be a historic city located abroad relative to the user's usual country of residence. Furthermore, the location of interest could be a historic landmark, a famous restaurant, a city center, a subway station, and / or any other location within the area. When the user is determined not to be within an area including a location of interest, method 400 can proceed to operation 416 via continuation element "B". Continuation element "B" can instruct operation 404 of method 400 to continue via continuation element "B" to operation 418 of method 420. Operation 418 can include generating response data based on spoken utterances received at the computing device. However, when the user is determined to be within an area including a location of interest, method 400 can proceed from operation 404 to operation 406.

[0072] Operation 406 can include location feature data that identifies the fulfillment of a request corresponding to a spoken utterance. The location feature data can be a dataset provided by one or more sources of data characterizing a region and / or location of interest. For example, the location feature data can include data from one or more websites describing a region and / or location of interest. Method 400 can proceed from operation 406 to operation 408, which can include assigning scores to each subset of the location feature dataset based on the extent or degree to which the subset fulfills the request. For example, when the spoken utterance includes natural language content such as “Assistant, how tall is it?”, the scores of a particular website detailing the dimensions of certain features of the location of interest can be prioritized over another website that does not include details about the dimensions of certain features of the location of interest—but otherwise provides information about the location of interest. In this way, data provided by a particular website can be considered a subset of the location feature data, and another website can be considered another subset of the location feature data. Based on the assigned scores, the subset of location feature data will therefore be prioritized over another subset of the location feature data.

[0073] Method 400 can proceed from operation 408 to operation 410, which can include determining whether the user's context satisfies criteria associated with the location of interest. For example, the criterion could be a threshold distance from the location of interest within the area. Therefore, the user's context satisfies the criterion when the user is located at or within the threshold distance. Alternatively or additionally, the criteria could be one or more characteristics of the user's context, such as: whether they are moving towards or away from the location of interest, whether they are above, below, or level with the location of interest, the arrival time the user will be able to reach the location of interest from their current location, whether the user has previously visited the location of interest, and / or any other property that can describe the user's context relative to the location of interest. When the user's context satisfies the criteria, method 400 can proceed to operation 412. However, when the user's context does not satisfy the criteria, method 400 can proceed from operation 410 to operation 414 of method 420 via continuation element "A".

[0074] Operation 412 can include modifying one or more scores of one or more subsets of location characteristic data corresponding to the location of interest. In other words, because the user's context satisfies the criteria, one or more subsets of location characteristic data corresponding to the location of interest can be assigned scores, and / or their corresponding scores can be modified to prioritize one or more subsets of location characteristic data. In this way, a subset of location characteristic data that takes precedence over all other subsets of location characteristic data can be used as the basis for providing a response output to the user in response to spoken utterances. Method 400 can proceed from operation 412 to operation 418 discussed herein via continuation element "B". However, as previously discussed, when the user's context does not satisfy the criteria associated with the location of interest, method 400 can proceed from operation 410 to operation 414.

[0075] Operation 414 may include determining whether the user's context satisfies other criteria associated with the region. For example, other criteria may be another threshold distance from the region, and thus the user's context satisfies geographical criteria when the user is located at or within another threshold distance. Alternatively or additionally, other criteria may be one or more characteristics of the user's context, such as whether they are moving toward or away from the region, whether they are above, below, or level with the region, the arrival time the user will be able to reach the region from their current location, whether the user has previously visited the region, and / or any other property that describes the user's context relative to the region.

[0076] When the user's context fails to meet other criteria associated with the region, method 420 can proceed from operation 414 to operation 418. However, when the user's context meets other criteria associated with the region, method 420 can proceed from operation 414 to operation 416. Operation 416 can include modifying one or more scores of one or more subsets of the location characteristic data corresponding to the region. In other words, because the user's context meets other criteria, one or more subsets of the location characteristic data corresponding to the region can be assigned scores, and / or their corresponding scores can be modified so that one or more subsets of the location characteristic data take precedence over other data associated with spoken discourse. Method 420 can proceed from operation 416 to operation 418, where operation 418 can include generating response data based on spoken discourse and / or a subset of location characteristic data that takes precedence over any other subset of the location characteristic data.

[0077] Method 420 can proceed from operation 418 to operation 422, whereby operation 422 can include rendering a response output based on the response data. The response output can be rendered by a computing device and / or an automated assistant guided by spoken utterances. For example, when the user's context meets criteria corresponding to a location of interest, the response output rendered in response to spoken utterances such as "Assistant, how tall is it?" can be based on a subset of location characteristic data representing a specific landmark corresponding to the location of interest. For example, when the user is within a threshold distance (e.g., 10 meters, 30 meters, and / or any other distance) of the Great Bhudda statue in Thailand, the automated assistant can provide a response output such as "The Great Bhudda statue is 92 meters tall."

[0078] However, when the user's context satisfies criteria corresponding to a region rather than a location of interest, the response output rendered in response to spoken utterances can be based on another subset of location-specific data. This other subset of location-specific data can characterize the highest point in the region and / or the size of one or more structures within the region. For example, when the user is located within the Thailand region but not within a threshold distance of the location of interest, the automated assistant can provide a different response output, such as "The tallest point in Thailand is Doi Inthanon, which is 2,565 meters tall." Alternatively, when the user is at home watching content on their computing device, the response output can be biased more towards the content being watched rather than towards a region that might be near the user or a location of interest.

[0079] For example, when a user is watching a television program about a Great Bhudda statue and provides spoken utterances (e.g., “How tall is it?”), the response output can be directed towards the content of the television program (e.g., “The Great Bhudda statue is 92 meters tall.”) rather than being biased towards location-specific data corresponding to the user’s current location. In this way, the user can provide a streamlined request based on the fact that the automated assistant can bias the processing of inputs and / or outputs according to the user’s location and / or other data that may be relevant to the user’s request. In particular, when a user is accessing a foreign region or location of interest, the user will be able to conserve computing resources at the computing device by reducing the average length of the spoken utterances provided to the corresponding automated assistant.

[0080] Figure 5 This is a block diagram of an example computer system 510. Computer system 510 typically includes at least one processor 514 that communicates with a number of peripheral devices via a bus subsystem 512. These peripheral devices may include a storage subsystem 524, which includes, for example, a memory subsystem 525 and a file storage subsystem 526, a user interface output device 520, a user interface input device 522, and a network interface subsystem 516. The input and output devices allow users to interact with computer system 510. The network interface subsystem 516 provides an interface to an external network and is coupled to corresponding interface devices in other computer systems.

[0081] User interface input device 522 may include a keyboard, pointing devices such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a touchscreen integrated into a display, audio input devices such as a voice recognition system, a microphone, and / or other types of input devices. Generally, the term "input device" is used to encompass all possible types of devices and methods for inputting information into computer system 510 or onto a communication network.

[0082] User interface output device 520 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for creating a visible image. The display subsystem may also provide non-visual displays, such as via an audio output device. Generally, the term "output device" is intended to encompass all possible types of devices and methods for outputting information from computer system 510 to a user or to another machine or computer system.

[0083] Storage subsystem 524 stores the programming and data construction functionality of some or all of the modules described herein. For example, storage subsystem 524 may include selected aspects and / or implementations of system 200, computing device 104, computing device 140, computing device 202, and / or any other applications, devices, apparatuses, and / or modules discussed herein for performing methods 300, 400, and 420.

[0084] These software modules are typically executed by processor 514 alone or in combination with other processors. The memory subsystem 525 used in storage subsystem 524 can include multiple memories, including main random access memory (RAM) 530 for storing instructions and data during program execution and read-only memory (ROM) 532 in which fixed instructions are stored. File storage subsystem 526 provides persistent storage for program and data files and may include hard disk drives, floppy disk drives, and associated removable media, CD-ROM drives, optical drives, or removable media cartridges. Modules implementing the functionality of certain implementations may be stored by file storage subsystem 526 in storage subsystem 524 or in other machines accessible by processor(s) 514.

[0085] Bus subsystem 512 provides a mechanism for allowing various components and subsystems of computer system 510 to communicate with each other as intended. Although bus subsystem 512 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0086] Computer systems 510 can be of various types, including workstations, servers, computing clusters, blade servers, server farms, or any other data processing system or computing device. Due to the constantly evolving nature of computers and networks, Figure 5 The description of the computer system 510 depicted herein is intended only as a specific example for illustrating some implementation methods. Many other configurations of the computer system 510 may have different characteristics. Figure 5 The computer system depicted in the text has more or fewer components.

[0087] In systems described herein that collect or may utilize personal information about users (or, as often referred to herein, "participants"), users may be provided with the opportunity to control whether a program or feature collects user information (e.g., information about a user's social networks, social actions or activities, occupation, user preferences, or the user's current geographic location) or to control whether and / or how content that may be more relevant to the user is received from a content server. Additionally, certain data may be processed in one or more ways before it is stored or used, such that personally identifiable information is removed. For example, a user's identity may be processed so that personally identifiable information for that user cannot be determined, or a user's geographic location may be generalized (e.g., down to the city, zip code, or state level) when geographic location information is obtained, making it impossible to determine the user's specific geographic location. Therefore, users can control how information about themselves is collected and / or used.

[0088] While several embodiments have been described and illustrated herein, various other means and / or structures may be utilized for performing functions and / or obtaining results and / or one or more advantages described herein, and each of such variations and / or modifications is considered to be within the scope of the embodiments described herein. More generally, all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications to which the teachings are applied. Those skilled in the art will recognize or be able to discover many equivalent forms of the particular embodiments described herein using more than routine experimentation. Therefore, it should be understood that the foregoing embodiments are presented by way of example only, and that embodiments may be practiced in ways other than those specifically described and claimed within the scope of the appended claims and their equivalents. Embodiments of this disclosure relate to each individual feature, system, article of manufacture, material, kit, and / or method described herein. Additionally, any combination of two or more such features, systems, articles of manufacture, materials, kits, and / or methods is included within the scope of this disclosure, provided that such features, systems, articles of manufacture, materials, kits, and / or methods are not inconsistent with each other.

[0089] In some embodiments, a method implemented by one or more processors is described as including operations such as: determining, based on data accessible via a portable computing device, that a user is within an area including a location of interest, wherein the location of interest is associated with location characteristic data accessible via one or more applications of the portable computing device, and wherein the portable computing device is operable in a first location-based bias mode and a second location-based bias mode. The method may further include operations such as: receiving spoken utterances from the user via an automation assistant of the portable computing device, wherein the spoken utterances include natural language content corresponding to a request to the automation assistant to provide information about the location of interest. When the portable computing device is operating in the first location-based bias mode, the method may, in response to receiving spoken utterances from the user, generate response data by biasing data selection from the location characteristic data according to a first bias degree, and render a response output based on the response data via the automation assistant of the portable computing device. The method may also include the following operations when the portable computing device is operating in a second location-based bias mode: in response to receiving spoken utterances from the user, generating additional response data by biasing another data selection from location characteristic data according to a second bias degree corresponding to a bias degree greater than a first bias degree, and rendering another response output based on the additional response data via the portable computing device's automatic assistant.

[0090] In some implementations, biasing another data selection from location characteristic data according to a second bias degree includes modifying the request by replacing the requested pronoun with an alias having a defined relationship with the location of interest. In some implementations, biasing another data selection from location characteristic data according to a first bias degree includes modifying the request by replacing the requested pronoun with an alias having a defined relationship with the region. In some implementations, the location characteristic data includes specific content characterizing features of the region and features of the location of interest. In some implementations, biasing another data selection from location characteristic data according to a second bias degree includes limiting data selection to a specific subset of the location characteristic data. In some implementations, biasing data selection from location characteristic data according to a first bias degree includes increasing the score assigned to the location characteristic data that satisfies the specific content of the request, and wherein generating additional response data is based on the score.

[0091] In some embodiments, biasing data selection from location characteristic data according to a first bias level includes increasing the score to a first limit for any specific content that satisfies the request. In some embodiments, biasing another data selection from location characteristic data according to a second bias level includes increasing the score to a second limit that has a greater magnitude than the first limit for any specific content that matches the request. In some embodiments, the method may further include, in response to determining that a user is in an area including a location of interest: causing a portable computing device to render a prompt asking the user whether they expect the automated assistant to enter a second location-based bias mode; and causing the computing device to operate according to the second location-based bias mode in response to receiving affirmative user input in response to the prompt. In some embodiments, determining that a location of interest is related to the user within the area is based on user-specific data mapped to a location of interest, wherein causing the portable computing device to render a prompt further in response to determining that the location of interest is related to the user within the area.

[0092] In some embodiments, the method may further include: determining, based on user-specific data mapped to a location of interest, that the location of interest is related to the user within a region; and, in response to determining that the user is within a region including the location of interest, and that the location of interest is related to the user within the region: automatically causing the computing device to operate according to a second location-based bias pattern. In some embodiments, biasing data selection from location characteristic data according to a first bias degree includes avoiding modification requests, and further biasing another data selection from location characteristic data according to a second bias degree includes modifying the request by adding an alias having a definitional relationship with the region.

[0093] In some embodiments, biasing data selection from location feature data according to a first bias degree includes biasing speech-to-text processing toward a first set of words associated with the location of interest, but not toward a second set of words also associated with the location of interest. Furthermore, biasing another set of data selection from location feature data according to a second bias degree includes biasing speech-to-text processing toward both the first and second set of words. In some embodiments, the method may further include selecting the first set of words based on its most frequent occurrence in one or more corpora associated with the location of interest.

[0094] In some implementations, the method may further include the following operations: determining whether the region or the location of interest was identified in assistant data generated during an interaction between the automatic assistant and the user before the user was located in the region including the location of interest, based on determining that the user is in the region including the location of interest, wherein when the region is identified in the assistant data, the computing device operates in a first location-based bias mode, and when the location of interest is identified in the assistant data, the computing device operates in a second location-based bias mode.

[0095] In other embodiments, a method implemented by one or more processors is described as including operations such as: determining, based on data accessible via a portable computing device, that a user is geographically located within an area including a location of interest, wherein the location of interest is characterized by location characteristic data accessible via one or more applications of the portable computing device. The method may further include operations such as: receiving spoken utterances from the user via an automated assistant on the portable computing device, wherein the spoken utterances include natural language content corresponding to a request to the automated assistant to provide information about the location of interest. The method may further include operations such as: assigning scores to each subset of location characteristic data based on the extent to which each subset of location characteristic data satisfies the limits of the request represented by the spoken utterances, and, when the user's geographic location satisfies geographic criteria associated with the location of interest: identifying the specific score assigned to the subset of location characteristic data representing the location of interest, and modifying the specific score assigned to the subset of location characteristic data, at least based on the subset of location characteristic data representing the location of interest. The method may further include the following operation: when the user's geographic location does not meet the geographic criteria associated with the location of interest, identifying another specific score assigned to another subset of location characteristic data representing the region, and modifying the other specific score assigned to the other subset of location characteristic data, at least based on the region represented by the other subset of location characteristic data. The method may further include the following operation: in response to receiving spoken utterance from the user, causing the automated assistant to render a response output via a portable computing device and based on the assigned score of the specific subset of location characteristic data having priority over other available subsets of location characteristic data.

[0096] In some embodiments, the method may further include modifying the request to include specific content based on a specific subset of location characteristic data, wherein the response output is based on the modified request. In some embodiments, the method may further include modifying the request to replace pronouns in the request with aliases having a defined relationship with the location of interest when the user's geographic location satisfies geographic criteria associated with the location of interest, wherein assigning a score to each subset of location characteristic data within the subset of location characteristic data is performed based on the extent to which each subset of location characteristic data satisfies the modified request.

[0097] In some embodiments, the method may further include the operation of modifying the request to replace the pronoun of the request with another alias having a different definitional relationship with the region when the user's geographic location does not meet the geographic criteria associated with the location of interest: wherein the assignment score for each subset of location characteristic data in a subset of location characteristic data is performed according to the limit that each subset of location characteristic data meets the modified request. In some embodiments, the method may further include the operation of causing the portable computing device to render a prompt asking the user to choose whether to operate the automated assistant according to a location-based bias pattern when the user's geographic location meets the geographic criteria associated with the location of interest: instructing the user to choose whether to operate the automated assistant according to a location-based bias pattern. In some embodiments, the method may further include the operation of receiving affirmative user input to prompt the automated assistant to operate according to a location-based bias pattern.

[0098] In another embodiment, a method implemented by one or more processors is described as including operations such as: receiving spoken utterances from a user located within a geographic area corresponding to a request to provide a response to an automated assistant, wherein, when the spoken utterances are received at a computing device, the automated assistant operates at the computing device according to a first location-based bias pattern. The method may further include operations such as: in response to receiving the spoken utterances, providing a response output according to the first location-based bias pattern, wherein the first location-based bias pattern causes the automated assistant to prioritize one or more subsets of location feature data when generating the response output, and wherein the response output identifies locations of interest within the area and is based on the highest priority subset of the location feature data. The method may further include operations such as: after the automated assistant provides the response output, determining that the user has relocated to a location of interest within the area. The method further includes the following operations: at least based on the user repositioning to a location of interest, causing the auto-assistant to operate according to a second location-based bias mode, different from a first location-based bias mode, wherein the second location-based bias mode causes the auto-assistant to prioritize one or more other subsets of location feature data when generating another response output, and wherein the one or more other subsets of location feature data are prioritized based on the degree of correspondence between the one or more other subsets of location feature data and the location of interest. The method may further include the following operations: while the auto-assistant is operating according to the second location-based bias mode, receiving additional spoken utterances corresponding to a separate request to the auto-assistant to provide information associated with the location of interest. The method may further include the following operations: in response to receiving the additional spoken utterances, providing another response output according to the second location-based bias mode, wherein the other response output is based on the most prioritized other subset of location feature data.

[0099] In some embodiments, one or more other subsets of location characteristic data are included in one or more subsets of location characteristic data. In some embodiments, the method may further include: at least based on the user relocating to a location of interest, causing a computing device to render a prompt soliciting the user's choice whether to switch the auto-assistant from a first location-based bias mode to a second location-based bias mode, wherein the auto-assistant operates according to the second location-based bias mode and provides an affirmative selection of the second location-based bias mode based on the user's response to the prompt. In some embodiments, providing a response output according to the first location-based bias mode includes generating a query comprising natural language content including spoken utterances and aliases of one or more words in the natural language content, wherein the aliases are selected based on one or more preferred subsets of location characteristic data.

[0100] In some implementations, providing another response output according to a second location-based bias pattern includes generating another query that includes other natural language content containing other spoken utterances and another alias for one or more other words in the other natural language content, wherein the other alias is selected based on one or more other subsets of location feature data. In some implementations, processing the input includes: for each subset of the location feature data, determining an embedding distance between an input embedding corresponding to a spoken utterance and an embedding corresponding to a specific subset of the location feature data, wherein the most preferred subset of the location feature data corresponds to the shortest embedding distance relative to other determined embedding distances. In some implementations, providing another response output according to a second location-based bias pattern includes: assigning one or more scores to one or more other subsets of the location feature data based on the degree of correspondence between one or more other subsets of the location feature data and other spoken utterances, and based on another degree of correspondence between one or more other subsets of the location feature data and a location of interest. In some implementations, the method may further include providing a supplementary response output according to a second location-based bias pattern, wherein the supplementary response output is pre-rendered proactively by an automatic assistant in response to the user repositioning to a location of interest within the region, and wherein the supplementary response output differs from the response output and another response output.

Claims

1. A method implemented by one or more processors, the method comprising: Based on data accessible via portable computing devices, the system determines the user's location within an area that includes their area of ​​interest. The location of interest is associated with location characteristic data that can be accessed via one or more applications of the portable computing device, and The portable computing device is capable of operating in a first location-based bias mode and a second location-based bias mode. The portable computing device's automated assistant receives spoken words from the user. The spoken discourse includes natural language content corresponding to a request to the automated assistant to provide information about the location of interest; When the portable computing device is operating in the first location-based bias mode: In response to receiving the spoken utterance from the user, response data is generated by biasing data selection from the location feature data according to a first bias degree, and The automatic assistant of the portable computing device renders the response output based on the response data; and When the portable computing device is operating in the second location-based bias mode: In response to receiving the spoken utterance from the user, additional response data is generated by biasing another data selection from the location feature data according to a second bias corresponding to a bias greater than the first bias. The automatic assistant of the portable computing device renders another response output based on the other response data.

2. The method according to claim 1, wherein, The other data selection, which biases the location characteristic data according to the second bias degree, includes modifying the request by replacing the pronoun in the request with an alias that has a defined relationship with the location of interest.

3. The method according to claim 1, wherein, The data selection based on the first bias degree biasing from the location characteristic data includes modifying the request by replacing the pronouns in the request with aliases that have a definitional relationship with the region.

4. The method according to claim 1, wherein, The location characteristic data includes specific content characterizing the features of the region and the location of interest.

5. The method according to claim 4, wherein, The other data selection based on the second bias degree includes limiting the data selection to a specific subset of the location characteristic data.

6. The method according to claim 5, wherein, The data selection based on the first bias degree biasing from the location characteristic data includes increasing the score assigned to the specific content from the location characteristic data that satisfies the request, and wherein the generation of the other response data is based on the score.

7. The method according to claim 4, wherein, The data selection based on the first bias degree biased from the location characteristic data includes increasing the score to a first limit for any specific content that satisfies the request.

8. The method according to claim 7, wherein, The other data selection based on the second bias degree, biasing the location characteristic data, includes, for any specific content matching the request, increasing the score to a second limit that has a greater magnitude than the first limit.

9. The method of claim 1, further comprising: In response to determining that the user is within the area including the location of interest: The portable computing device renders a prompt asking the user whether they expect the automated assistant to enter a second location-based bias mode; as well as The portable computing device is made to operate according to the second location-based bias mode in response to receiving affirmative user input in response to the prompt.

10. The method of claim 9, further comprising: Based on user-specific data mapped to the location of interest, it is determined that the location of interest is related to the user within the region. The portable computing device renders the prompt in response to determining that the location of interest is relevant to the user within the area.

11. The method of claim 1, further comprising: Based on user-specific data mapped to the location of interest, it is determined that the location of interest is related to the user within the region; as well as In response to determining that the user is within the area including the location of interest, and determining that the location of interest is related to the user within the area: The portable computing device is automatically made to operate according to the second location-based bias mode.

12. The method according to claim 4, in, The data selection based on the first bias degree biased from the location characteristic data includes avoiding modification of the request, and The other data selection, which biases the location characteristic data according to the second bias degree, includes modifying the request by adding an alias that has a definitional relationship with the region.

13. The method according to claim 1, in, The data selection based on the first bias degree, derived from the location characteristic data, includes biasing the speech-to-text processing toward a first set of words related to the location of interest, but not toward a second set of words also related to the location of interest. The other data selection, which biases the location feature data according to the second bias degree, includes biasing speech-to-text processing toward the first word set and toward the second word set.

14. The method of claim 13, further comprising: The first word set is selected based on the most frequent occurrence of the first word set in one or more corpora associated with the location of interest.

15. The method according to any one of claims 1-14, further comprising: Based on determining that the user is within the area including the location of interest, it is determined whether the area or the location of interest was identified in assistant data generated during the interaction between the automated assistant and the user before the user was within the area including the location of interest. Specifically, when the region is identified in the assistant data, the portable computing device operates in the first location-based bias mode, and when the location of interest is identified in the assistant data, the portable computing device operates in the second location-based bias mode.

16. A method implemented by one or more processors, the method comprising: Based on data accessible via portable computing devices, it is determined that the user is geographically located within an area that includes the location of interest. The location of interest is characterized by location characteristic data that can be accessed via one or more applications of the portable computing device; The portable computing device's automated assistant receives spoken words from the user. The spoken discourse includes natural language content corresponding to a request to the automated assistant to provide information about the location of interest; Assign a score to each subset of the location feature data based on the degree to which each subset of the location feature data satisfies the request represented by the spoken discourse, and When the user's geographic location meets the geographic criteria associated with the location of interest: Identify and assign a specific score to a subset of the location characteristic data representing the location of interest, and The location of interest is characterized based on at least a subset of the location characteristic data, such that the specific score assigned to the subset of the location characteristic data is modified. When the user's geographic location does not meet the geographic criteria associated with the location of interest: Identify and assign another specific score to another subset of the locational characteristics data characterizing the region, and The region is characterized based on at least another subset of the location characteristic data, such that the other specific score assigned to the other subset of the location characteristic data is modified; and In response to receiving the spoken utterance from the user, and based on a score assigned to a specific subset of the location feature data that takes precedence over other available subsets of the location feature data, the automated assistant renders a response output via the portable computing device.

17. The method of claim 16, further comprising: The request is modified to include specific content based on the specific subset of the location characteristic data, wherein the response output is based on the modified request.

18. The method of claim 16, further comprising: When the user's geographic location satisfies the geographic criteria associated with the location of interest: The request is modified to replace the pronouns in the request with aliases that have a defined relationship with the location of interest. The assignment of the score to each subset of the location characteristic data within the subset of location characteristic data is performed based on the limit that each subset of the location characteristic data satisfies the modified request.

19. The method of claim 16, further comprising: When the user's geographic location does not meet the geographic criteria associated with the location of interest: The request is modified to replace the pronouns in the request with another alias that has a different definitional relationship with the region. The assignment of the score to each subset of location characteristic data within a subset of location characteristic data is performed based on the limit that each subset of location characteristic data satisfies the modified request.

20. The method according to any one of claims 16-19, further comprising: When the user's geographic location meets the geographic criteria associated with the location of interest: The portable computing device renders a prompt asking the user to choose whether to operate the automated assistant based on a location-based bias pattern; as well as The system receives positive user input to prompt the automated assistant to operate according to the location-based bias pattern.

21. A method implemented by one or more processors, the method comprising: Receive spoken words from users located within a geographic area that correspond to requests for a response to the automated assistant. When the spoken words are received at the computing device, the automatic assistant operates at the computing device according to a first location-based bias mode. In response to receiving the spoken utterance, a response output is provided according to the first location-based bias pattern. Wherein, the first location-based bias mode causes the automatic assistant to prioritize one or more subsets of location characteristic data when generating the response output, and The response output identifies the location of interest within the region and is based on the highest priority subset of the location characteristic data; After the automatic assistant provides the response output, it is determined that the user has relocated to the location of interest within the area; At least based on the user repositioning to the location of interest, the automatic assistant operates according to a second location-based bias mode, which is different from the first location-based bias mode. The second location-based bias mode causes the automatic assistant to prioritize one or more other subsets of the location characteristic data when generating another response output, and Wherein, one or more other subsets of the location characteristic data are prioritized based on the degree of correspondence between the one or more other subsets of the location characteristic data and the location of interest; While the automated assistant is operating according to the second location-based bias pattern, another spoken utterance is received corresponding to a separate request to the automated assistant to provide information associated with the location of interest; and In response to receiving the other spoken utterance, the other response output is provided according to the second location-based bias pattern. The other response output is based on the highest priority subset of the location characteristic data.

22. The method according to claim 21, wherein, The one or more other subsets of the location characteristic data are included in the one or more subsets of the location characteristic data.

23. The method of claim 21, further comprising: At least based on the user repositioning to the location of interest, the computing device renders a prompt asking the user to choose whether to switch the automatic assistant from the first location-based bias mode to the second location-based bias mode. The automatic assistant operates according to the second location-based bias mode based on the user's positive selection of the second location-based bias mode in response to the prompt.

24. The method according to claim 21, in, Providing the response output based on the first location-based bias pattern includes generating a query that includes natural language content of the spoken utterance and aliases of one or more words in the natural language content, and The alias is selected based on one or more priority subsets of the location characteristic data.

25. The method according to claim 21, in, Providing the other response output based on the second location-based bias pattern includes generating another query that includes other natural language content containing the other spoken utterance and another alias for one or more other words in the other natural language content, and The other alias is selected based on one or more other subsets of the location characteristic data.

26. The method according to claim 21, wherein, Input processing includes: For each subset of one or more subsets of the location feature data, determine the embedding distance between the input embedding corresponding to the spoken utterance and each embedding corresponding to a specific subset of one or more subsets of the location feature data. The highest priority subset of the location characteristic data corresponds to the shortest embedding distance relative to other determined embedding distances.

27. The method according to claim 21, wherein, Another response output based on the second location-based bias pattern includes: Based on the degree of correspondence between one or more other subsets of the location feature data and the other spoken discourse, and based on another degree of correspondence between one or more other subsets of the location feature data and the location of interest, one or more scores are assigned to one or more other subsets of the location feature data.

28. The method according to any one of claims 21-27, further comprising: Supplementary response output is provided based on the second position-based bias pattern. The supplementary response output is pre-rendered proactively by the automatic assistant in response to the user repositioning to the location of interest within the area, and the supplementary response output is different from the response output and the other response output.

29. A computer program product comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of the preceding claims.

30. A computer-readable storage medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 28.

31. A system comprising one or more processors for performing the method according to any one of claims 1 to 28.

Images