Method for providing a prompt in an automated dialog session based on selected content from previous automated dialog sessions, client device and computer program configured to perform the method

Abstract

A method implemented by one or more processors and comprising: as part of a dialogue session between a user and an automated assistant implemented by one or more of the processors: Receiving natural language input based on user interface input provided by the user via a user interface input device; and Providing response content in response to natural language input as a response from the automatic assistant to the natural language input, where the content is provided to the user for presentation via a user interface output device and wherein the content contains at least one content parameter which is selected by the automated assistant from several candidate content parameters; as part of an additional dialogue session between the user and the automated assistant, which is temporally separate from the dialogue session: Providing a prompt that requests feedback regarding the selected content parameters; wherein the prompt to present is provided to the user via the user interface output device or an additional user interface output device, and where the prompt is generated based on the fact that the content parameter was previously selected by the automated assistant and previously made available for presentation to the user as part of the dialog session, in order to request feedback regarding the content parameter, based on providing the prompt, preemptively activating a microphone designed to process the user interface inputs provided via the microphone, wherein the user interface input device or the additional user interface input device includes the microphone, and Receiving additional input in response to the prompt, wherein the additional input is based on additional user interface input provided by the user via the microphone; and Using the additional inputs to influence a value stored in association with the content parameter, wherein the value stored in association with the content parameter influences the future provision of further content containing the content parameter.

Description

background

[0001] Users can interact with automated assistants (also known as "personal assistant modules," "mobile assistants," or "chatbots") via a variety of data processing devices, such as smartphones, tablet computers, wearable devices, automotive systems, standalone personal assistant devices, and so on. The automated assistants receive input from the user (e.g., typed and / or spoken natural language input) and respond with content (e.g., visual and / or audible natural language output).

[0002] US 2014 / 0114705 A1 discloses a computer-implemented method for travel planning for a traveler. The method includes establishing a travel plan for the traveler and performing a search using a processor for travel options that match the travel plan and the traveler's preferences, as well as storing the available travel options in one or more memory locations. Subsequently, one or more travel options are selected from the available travel options. In some situations, the one or more travel options are selected without the traveler's involvement. The traveler can then be informed of the one or more selected travel options.Other publications include US 2016 / 0 110 347 A1, which relates to systems and procedures for providing responses to natural language input, and US 2016 / 0 260 436 A1, which relates generally to intelligent automated assistants and specifically to techniques for triggering intelligent automated assistants. Summary

[0003] Claim 1 defines a method for providing a prompt in an automated dialogue session based on selected content from previous automated dialogue sessions, claim 19 defines a client device configured to execute the method, and claim 18 defines a computer program. In response to input from a user during a dialogue session, an automated assistant can provide a suggestion and / or other content in response to the input, which includes one or more content parameters selected by the automated assistant from several candidate content parameters based on one or more factors. For example, in response to the input "Where can I get a good burger?", the automated assistant can identify several candidate restaurants that serve burgers, but it can only select a subset (e.g.,Select one of these to include in the response content in response to the input.

[0004] This description refers to methods, devices, and computer-readable media that involve requesting feedback from a user regarding one or more content parameters of a suggestion or other content provided by the automated assistant. The user's feedback can be used to influence future suggestions and / or other content subsequently delivered by the automated assistant to the user and / or other users in future dialogue sessions.

[0005] In some implementations, content is delivered to the user by an automated assistant in a dialog session between the user and the automated assistant. The automated assistant then provides a prompt requesting user feedback regarding the delivered content in a subsequent dialog session. In some of these implementations, the prompt is provided after user input and / or output from the automated assistant in the subsequent dialog session that is unrelated to the content delivered in the previous dialog session.

[0006] As an example, the future dialogue session can be initiated by the user, prompting the automated assistant to perform a "routine action," such as providing a summary of the user's calendar entries, delivering messages, playing music, and so on. The automated assistant can perform some or all of the routine actions and then provide the prompt. This can add variety to the routine actions and / or enhance the user experience by introducing a conversational element into the routine actions.Furthermore, as mentioned above, feedback (direct or indirect) from a user provided in response to the prompt can be used to influence future suggestions or other content subsequently presented to the user by the automated assistant – thereby increasing the likelihood that the subsequent content delivered to the user by the automated assistant will be relevant to the user.Such an improvement in the relevance of the content provided by the automated assistant can enable the desired content to be provided to the user more quickly, which can reduce various computing resources (and then, if the automated assistant is not integrated into the client device, communication resources) that would otherwise be needed in a lengthy dialogue to identify the desired content.

[0007] In some implementations, a procedure executed by one or more processors is provided, which, as part of a dialogue session between a user and an automated assistant implemented by one or more of the processors, includes: receiving natural language input based on user interface input provided by the user via a user interface input device; and providing response content in response to the natural language input as a response from the automated assistant. The content is provided for presentation to the user via a user interface output device and contains at least one content parameter selected by the automated assistant from several candidate content parameters.The procedure further includes, as part of an additional dialogue session between the user and the automated assistant, which is temporally separate from the dialogue session: providing a prompt requesting feedback regarding the selected content parameters; receiving additional input in response to the prompt; and using the additional input to influence a value stored in connection with the content parameter. The value stored in connection with the content parameter influences the future provision of further content containing the content parameter. The prompt is provided for presentation to the user via the user interface output device or an additional user interface output device.The prompt is generated based on the content parameter previously selected by the automated assistant and presented to the user as part of the dialog session, requesting feedback regarding that parameter. Additional input in response to the prompt is based on additional user interface input provided by the user via the user interface input device or an additional user interface input device.

[0008] These and other implementations of the technology disclosed herein may optionally include one or more of the following features.

[0009] In some implementations, the user initiates the additional dialog session with additional natural language input that is unrelated to the content of the previous dialog session. In some of these implementations, the procedure further includes, as part of the additional dialog session: providing additional dialog session output in response to the additional natural language input that is unrelated to the content of the previous dialog session. Providing the prompt may occur after providing the additional dialog session output. In some versions of these implementations, the procedure further includes determining that one or more criteria are satisfied by the additional natural language input and / or the additional dialog session output. Providing the prompt may also be based on determining that the criteria are satisfied.The criteria may include, for example, that the additional dialog session outputs are of a certain semantic type, and / or that at least one N-gram of a set of criteria N-grams occurs in the user prompt.

[0010] In some implementations, the content is a suggestion that the user should respond to in the future, and the procedure further includes determining that the user has responded to the suggestion after the content containing the content parameter has been provided. Providing the prompt may also be based on determining that the user has responded to the suggestion.

[0011] In some implementations, the user interface input device or the additional user interface input device used to generate the additional inputs in response to the prompt includes a microphone, and the method further includes: based on the provision of the prompt, preemptively activating at least one component designed to process the user interface inputs provided via the microphone.

[0012] In some implementations, the user initiates the additional dialog session, and the provision of the prompt depends on the user having initiated the additional dialog session.

[0013] In some implementations, the procedure also includes, as part of the additional dialog session: providing additional dialog session output before providing the prompt. Providing the prompt can occur after providing the additional dialog session output.

[0014] In some implementations, the procedure further includes: identifying an additional content parameter that is provided to the user as part of an additional prior dialogue session between the user and the automated assistant; and determining, based on one or more criteria, to provide the prompt based on the content parameter instead of an alternative prompt based on the additional content parameter. In some of these implementations, the one or more criteria include a corresponding temporal proximity between the provision of the content parameter and the provision of the additional content parameter. In some of these implementations, the one or more criteria additionally or alternatively include semantic types associated with the content parameter and the additional content parameter.

[0015] In some implementations, the content includes a suggestion regarding a specific physical location and a specific item consumable at that location, and the content parameter identifies the specific item.

[0016] In some implementations, the content contains a suggestion regarding a specific physical location, and the content parameter identifies a category to which that specific physical location belongs.

[0017] In some implementations, both the input and additional input are generated via the user interface input device, and both the content and the feedback prompt are provided for presentation via the user interface output device. In some of these implementations, the user interface input device includes a microphone of a single device, and the user interface output device includes a speaker of a single device.

[0018] In some implementations, the inputs are generated via the user interface input device of a first computing device, the content is provided for presentation via the user interface output device of the first computing device, and the prompt is provided for presentation via the additional user interface output device of an additional computing device.

[0019] In some implementations, a procedure performed by one or more processors is provided that includes: identifying a stored content parameter of content previously provided to a user from a computer-readable medium as part of a prior dialog session between a user and an automated assistant implemented by one or more of the processors. The procedure further includes, as part of an additional dialog session between the user and the automated assistant, temporally separate from the dialog session: providing a prompt requesting feedback regarding the content parameter.The prompt is presented to the user via a user interface output device of the user's computing device, and the prompt is generated based on the fact that the content parameter was previously presented to the user as part of the preceding dialog session, in order to request feedback regarding the content parameter. The procedure further includes, as part of the additional dialog session: receiving additional input in response to the prompt; and using the additional input to modify a value stored in connection with the content parameter.The additional inputs are based on additional user interface inputs provided by the user via a user interface input device of the computing device, and the stored value influences the future provision of further content containing the content parameter.

[0020] Additionally, some implementations include one or more processors from one or more computing devices, wherein the one or more processors are operable to execute instructions stored in an allocated memory, and wherein the instructions are designed to cause one of the aforementioned procedures to be carried out. Some implementations also include one or more non-transient, computer-readable storage media that store computer instructions which can be executed by one or more processors to carry out one of the aforementioned procedures.

[0021] It should be noted that all combinations of the preceding concepts and additional concepts described in more detail herein are considered part of the subject matter disclosed here. For example, all combinations of claimed items appearing at the end of this disclosure are considered part of the subject matter disclosed here. Brief description of the drawings Fig. Figure 1 is a block diagram of an exemplary environment in which implementations disclosed herein can be implemented. Fig. Figure 2A shows an example of a dialog session in which the automated assistant provides content to the user with content parameters selected from several candidate content parameters. Fig. 2B 1, Fig. 2B2, Fig. 2B3 and Fig. 2B4 shows various examples of providing a feedback request based on one or more selected content parameters of Fig. 2A based, for the user in a dialog session that differs from that of Fig. 2A is separated. Fig. Figure 3A shows another example of a dialog session where the automated assistant provides content to the user with content parameters selected from several candidate content parameters. Fig. 3B shows an example of deploying a feedback request based on one or more selected content parameters of Fig. 3A based, for the user in a dialog session that differs from that of Fig. 3A is separated. Fig. Figure 4 is a flowchart that illustrates an exemplary procedure according to the implementations disclosed herein. Fig. Figure 5 shows an exemplary architecture of a computing device. Detailed description

[0022] With reference to Fig. Figure 1 shows an exemplary environment in which the techniques disclosed herein can be implemented. The exemplary environment contains several client computing devices 1061-N and an automated assistant 120. Although the automated assistant 120 is in Fig. As shown in Figure 1, separate from the client computing devices 1061-N, in some implementations the entire Automated Assistant 120, or aspects thereof, may be implemented by one or more of the client computing devices 1061-N. For example, a client device 1061 may implement one instance or multiple aspects of the Automated Assistant 120, and a client device 106N may also implement a separate instance of this one or more of these aspects of the Automated Assistant 120. In implementations where one or more aspects of the Automated Assistant 120 are implemented by one or more computing devices located remotely from the client computing devices 1061-N, the client computing devices 1061-N and these aspects of the Automated Assistant may communicate over one or more networks, such as a local area network (LAN) and / or a wide area network (WAN) (e.g., the Internet).

[0023] The 1061-N client computing devices may, for example, include one or more of the following: a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device in the user's vehicle (e.g., an in-vehicle communication system, an in-vehicle entertainment system, an in-vehicle navigation system), and / or a user-worn device containing a computing device (e.g., a user's watch with a computing device, user's glasses with a computing device, a virtual or augmented reality computing device). Additional and / or alternative client computing devices may be provided.In some implementations, a specific user can communicate with the automated assistant 120 using multiple client computing devices that collectively form a coordinated "ecosystem" of computing devices. However, for the sake of brevity, some examples described in this document will focus on a user operating a single client computing device 106.

[0024] Each of the 1061-N client computing devices can run a variety of different applications, such as a corresponding 1071-N message exchange client. The 1071-N message exchange clients can take various forms, and these forms can vary across the 1061-N client computing devices, and / or multiple forms can run on a single 1061-N client computing device. In some implementations, one or more of the 1071-N message exchange clients may take the form of an instant messaging service (“SMS”) and / or a multimedia messaging service (“MMS”), an online chat service (e.g., instant messaging, Internet Relay Chat, or “IRC,” etc.), a messaging application associated with a social network, a messaging service for personal assistants dedicated to conversations with the automated assistant 120, and so on.In some implementations, one or more of the message exchange clients 1071-N may be implemented via a website or other resources rendered by a web browser (not shown) or another application of the client computing device 106.

[0025] As described in more detail herein, the Automated Assistant 120 participates in dialog sessions with one or more users via user interface input devices and user interface output devices from one or more client computing devices 1061-N. In some implementations, the Automated Assistant 120 can participate in a dialog session with a user in response to user interface inputs provided by the user via one or more user interface input devices of one of the client computing devices 1061-N. In some of these implementations, the user interface is explicitly directed toward the Automated Assistant 120.For example, one of the 1071-N message exchange clients can be a personal assistant messaging service dedicated to conversations with the automated assistant 120, and the user interface inputs provided through this personal assistant messaging service can be automatically delivered to the automated assistant 120. Likewise, the user interface inputs in one or more of the 1071-N message exchange clients can be explicitly directed to the automated assistant 120, based on specific user interface inputs indicating that the automated assistant 120 should be invoked. For example, the specific user interface inputs can consist of one or more typed characters (e.g., @automatedassistant), a user interaction with a hardware button and / or a virtual button (e.g.,This may consist of a tap, a long tap, a verbal command (e.g., "Hey Automated Assistant"), and / or other specific user interface inputs. In some implementations, Automated Assistant 120 can participate in a dialogue session in response to user interface inputs even if those inputs are not explicitly directed at Automated Assistant 120. For example, Automated Assistant 120 may examine the contents of the user interface inputs and participate in a dialogue session in response to specific terms present in the inputs or based on other clues.

[0026] Each of the Client Computing Devices 1061-N and the Automated Assistant 120 can contain one or more memories for storing data and software applications, one or more processors for accessing data and executing applications, and other components that enable communication over a network. The operations performed by the one or more Client Computing Devices 1061-N and / or by the Automated Assistant 120 can be distributed across multiple computer systems. The Automated Assistant 120, for example, can be implemented as computer programs that run on one or more computers at one or more locations connected to each other by a network.

[0027] The automated assistant 120 may include a natural language processor 122, a response content engine 130, and a feedback engine 140. In some implementations, one or more of the engines and / or modules of the automated assistant 120 may be omitted, combined, and / or implemented in a component separate from the automated assistant 120. The automated assistant 120 participates in dialogue sessions with one or more users via associated client devices 1061-N to provide response content generated by the response content engine 130 and / or to provide feedback requests generated by the feedback engine 140.

[0028] In some implementations, the Response Content Engine 130 generates response content in response to various inputs provided by a user of one of the Client Computing Devices 1061-N during a dialog session with the Automated Assistant 120. The Response Content Engine 130 provides the response content to the user for presentation as part of the dialog session (e.g., over one or more networks if it is separate from a user's Client Computing Device). For example, the Response Content Engine 130 can generate response content in response to free-form natural language input provided through one of the Client Computing Devices 1061-N. As used here, free-form input is input formulated by a user and is not restricted to a set of options presented to the user for selection.

[0029] In response to certain inputs, the Response Content Engine 130 can generate content with one or more content parameters chosen from several candidate content parameters. For example, the provided input might be "give me directions to a good cafe," and the Response Content Engine 130 can first determine directions to a specific cafe by selecting a "good cafe" from several available cafes based on one or more factors. Furthermore, the Response Content Engine 130 can determine the directions by selecting from several candidate directions (e.g., shortest vs. fastest; include vs. exclude highways) based on one or more factors. An additional description of Response Content Engine 130 implementations is provided below.

[0030] In some implementations, the Feedback Engine 140 stores various content parameters selected by the Response Content Engine 130 and presented to a user in a dialog session, and generates a prompt requesting feedback from the user regarding one or more of the selected content parameters. The Feedback Engine 140 presents the feedback prompt to the user as part of a dialog session. In some implementations, the Feedback Engine 140 presents a feedback prompt to the user as part of a dialog session separate from the dialog session in which one or more of the content parameters that are the focus of the feedback prompt are presented to the user.The feedback engine 140 can also use feedback provided by the user in response to the prompt to influence future suggestions and / or other content subsequently provided by the automated assistant 120 to the user and / or other users in future dialog sessions.

[0031] As mentioned above, in some implementations, the automated assistant 120 provides content to the user during a dialog session between the user and the automated assistant. In a subsequent dialog session, the automated assistant 120 then provides a feedback prompt requesting user feedback regarding the provided content. In some of these implementations, the prompt for user input and / or output from the automated assistant 120 in the subsequent dialog session is unrelated to the content provided in the previous dialog session.

[0032] As used herein, a "dialogue session" can comprise a logically self-contained exchange of one or more messages between a user and the automated assistant 120. The automated assistant 120 can switch between multiple dialogue sessions with a user based on various signals, such as the time elapsed between sessions, a change in the user context (e.g., location, before / during / after a scheduled meeting, etc.) between sessions, or the detection of one or more intervening interactions between the user and a client device in addition to the dialogue between the user and the automated assistant (e.g.,This differs from situations where the user switches applications for a while, the user moves away from a standalone voice-enabled product and then returns later, locking / sleeping the client device between sessions, changing the client devices used to communicate with one or more instances of the automated assistant, and so on.

[0033] In some implementations, when Automated Assistant 120 provides a prompt requesting user feedback, it may proactively activate one or more components of the client device (through which the prompt is provided) that are designed to process user interface input to be received in response to the prompt.For example, if user interface input is to be provided via a microphone on client device 1061, the automated assistant 120 can provide one or more commands to: proactively "open" the microphone (thus avoiding the need to press an interface element or say a "hot word" to open the microphone), proactively activate a local speech-to-text processor on client device 1061, proactively establish a communication session between client device 1061 and a remote speech-to-text processor, and / or render a graphical user interface (e.g., an interface containing one or more selectable elements that can be chosen to provide feedback) on client device 1061.This can make it possible to provide and / or process user interface inputs faster than if the components are not activated preventively.

[0034] In some implementations, the automated assistant 120 can provide a prompt requesting user feedback in a future dialogue session based on whether user input and / or response output from the automated assistant 120 in that future dialogue session meet one or more criteria. For example, a prompt can be provided only if the user initiates the dialogue session with any user input and / or with specific user input, such as any natural language input and / or specific natural language input. Similarly, a prompt can be provided only if the user input and / or response output consists of one or more specific semantic types and / or does not consist of one or more specific semantic types.As another example, a prompt can be provided only if the user input and / or response outputs contain one or more specific N-grams and / or do not contain one or more other specific N-grams. In some implementations, the criteria can be selected to increase the likelihood of providing the prompt when the dialogue of a dialog session is conversational and / or light, and / or to decrease the likelihood of providing the prompt when the dialogue of the dialog session is task-oriented (e.g., to prevent distracting the user from the task).

[0035] In some implementations, criteria can be selected additionally or alternatively to protect the privacy of the user for whom the prompt is being provided. For example, when providing voice-based natural language input, the criteria might be that the voice-based input matches a user's voice profile and / or that the voice-based input does not contain background noise (e.g., the absence of background noise that might indicate other users are present). Similarly, the criteria might be that the user input and / or response outputs contain content that is personal to the user (e.g., response outputs that provide a summary of the user's calendar entries for the day), which could indicate that the user is in an environment they consider private.

[0036] In some implementations, content parameters from several different suggestions presented to the user may be available to the Feedback Engine 140 for generating a prompt to present to the user in a given future dialog session. In some of these implementations, the Feedback Engine 140 may select a subset (e.g., one) of these multiple suggestions to present in a given prompt based on one or more criteria. For example, whether a content parameter of a particular suggestion is used to generate and / or present a prompt may depend on a category of that particular suggestion (e.g., food suggestions are more likely to be selected than music suggestions), a subcategory of that particular suggestion (e.g.,Suggestions for French cuisine are more likely to be selected than suggestions for Mexican cuisine), a time at which the specific suggestion was provided (e.g., more recent suggestions are more likely to be selected than less recent suggestions), and / or a category of the specific suggestion and a time at which the specific suggestion was provided (e.g., a music suggestion from three days ago cannot be provided, but a food suggestion from three days ago can).

[0037] In some implementations, a prompt requesting feedback on a content parameter of a provided suggestion, which the user is expected to respond to in the future, can be provided by the automated assistant 120 based on a determination that the user has indeed responded to the suggestion. This can reduce the risk of prompting the user for feedback on a content parameter with which the user has never interacted, which can be confusing for the user and / or unnecessarily consumes computing resources when providing the prompt. In some implementations, a determination that a user has responded to the suggestion can be based on the dialogue between the user and the automated assistant 120 in a dialog session that occurred prior to the dialog session in which the prompt is provided (e.g.,Dialogue through which the user actually acted on the suggestion). In some implementations, determining that a user acted on the suggestion can be based on one or more additional or alternative signals, such as signals not generated by using Automated Assistant 120 to act on the suggestion. For example, suppose Automated Assistant 120 provides the user with a suggestion to visit a cafe. Before a prompt requesting feedback about the cafe is presented to the user in a future dialog session, it can first be determined, based on location data from the user's mobile client device, transaction data associated with the user, and / or other signals, that the user actually visited the cafe.

[0038] The Natural Language Processor 122 of the Automated Assistant 120 processes natural language input generated by the user via the Client Devices 1061-N and produces annotated output for use by one or more components of the Automated Assistant 120, such as the Response Content Engine 130 and / or the Feedback Engine 140. For example, the Natural Language Processor 122 can process free-form natural language input generated by the user via one or more user interface input devices of the Client Device 1061. The generated annotated output includes one or more annotations about the natural language input and optionally one or more (e.g., all) terms from the natural language input.

[0039] In some implementations, the Natural Language Processor 122 is designed to identify and annotate various types of grammatical information in natural language input. For example, the Natural Language Processor 122 may include a language marker designed to annotate terms with their grammatical role. For example, the language marker can tag each term with its sentence part, such as "noun," "verb," ​​"adjective," "pronoun," etc. Similarly, in some implementations, the Natural Language Processor 122 may additionally and / or alternatively include a dependency parser designed to determine syntactic relationships between terms in natural language input. For example, the dependency parser can determine which terms modify other terms, subjects and verbs of sentences, and so on (e.g.,a parser tree) and can make notes on such dependencies.

[0040] In some implementations, the natural language processor 122 may additionally or alternatively include an entity marker designed to record entity references in one or more segments, such as references to people, organizations, locations, and so on. The entity marker can record references to an entity at a very coarse level (e.g., to identify all references to an entity class such as people) and at a very fine level (e.g., to identify all references to a specific entity such as a particular person). The entity marker can rely on the content of the natural language input to clarify a particular entity and / or can optionally communicate with a knowledge graph or another entity database (e.g., a content database 152) to clarify a particular entity.

[0041] In some implementations, the Natural Language Processor 122 may additionally and / or alternatively include a coreference resolver designed to group or "cluster" references to the same entity based on one or more contextual clues. For example, the coreference resolver can be used to resolve the term "it" in the natural language input "I like the stir-fry at Asian Village. Please order it!" as "Asian Village".

[0042] In some implementations, one or more components of the Natural Language Processor 122 can rely on notes from one or more other components of the Natural Language Processor 122. For example, in some implementations, the mentioned entity marker, when noting all mentions of a particular entity, can rely on notes from the coreference resolver and / or the dependency parser. Similarly, in some implementations, the coreference resolver, for example, when grouping references to the same entity, can rely on notes from the dependency parser. In some implementations, when processing certain natural language inputs, one or more components of the Natural Language Processor 122 can use related prior inputs and / or other related data in addition to the certain natural language inputs to determine one or more notes.

[0043] As mentioned above, the response content engine 130 uses one or more resources when generating suggestions and / or other content for delivery during a dialog session with a user of one of the client devices 1061-N. The response content engine 130 may include an action module 132, an entity module 134, a content generation module 136, and an attribute module 138.

[0044] The Action Module 132 of the Response Content Engine 130 uses natural language input received from the Client Computing Devices 1061-N and / or annotations to the natural language input provided by the Natural Language Processor 122 to determine at least one action with respect to the natural language input. In some implementations, the Action Module 132 can determine an action based on one or more terms contained in the natural language input. For example, the Action Module 132 can determine an action based on the fact that the action on one or more computer-readable media is associated with one or more terms contained in the natural language input. For example, an action "make a reservation at a restaurant" can be associated with one or more terms such as "book," "reserve," "reservation," "get me a table," etc.Similarly, an action like "Provide a daily briefing" can be assigned to one or more terms such as "Tell me about my day," "What's happening today," "Good morning," etc. As another example, an action like "Provide a chat" can be assigned to one or more terms such as "Hello," "What's up?" etc.

[0045] In some implementations, the action module 132 can determine an action, at least in part, based on one or more candidate entities determined by the entity module 134 based on the natural language input of a dialogue session. For example, suppose the natural language input "Book it!" occurs, and "Book it!" is associated with several different actions, such as "make a restaurant reservation," "make a hotel reservation," "arrange a date," etc. In such a situation, the action module 132 can determine which action is the correct one based on the candidate entities determined by the entity module 134. For example, if the entity module 134 identifies only several restaurants as candidate entities, the action module 132 can determine that the correct action is "make a restaurant reservation."

[0046] The entity module 134 determines candidate entities based on input provided by one or more users during a dialogue session between the one or more users and the automated assistant 120 via one or more user interface input devices. The entity module 134 uses one or more resources when determining candidate entities and / or refining these candidate entities. For example, the entity module 134 can use the natural language input itself, annotations provided by the natural language processor 122, one or more attributes provided by the attribute module 138, and / or content from the content database 152.

[0047] The content database 152 can be provided on one or more non-transient, machine-readable media and can define multiple entities, properties of each entity, and optionally, relationships between these entities. For example, the content database 152 can contain an identifier for a specific restaurant and one or more properties of that restaurant, such as location, cuisine type(s), available dishes, opening hours, alternative names, whether reservations are accepted, a rating of that restaurant, a price, etc. The content database 152 can additionally or alternatively contain multiple protocols, each applicable to one or more entities and / or one or more actions.For example, each protocol can define one or more necessary and / or desired content parameters to perform an associated action and / or to perform the associated action with one or more associated entities.

[0048] The Content Generation Module 136 participates in a dialogue with one or more users via associated client devices to generate suggestions for performing an action and / or to generate other content. The Content Generation Module 136 optionally uses one or more resources when generating the content. For example, the Content Generation Module 136 can use current and / or past natural language input from a user during a dialogue session, annotations to that input provided by the Natural Language Processor 122, one or more attributes provided by the Attributes Module 138, one or more entities determined by the Entity Module 134, and / or one or more actions determined by the Actions Module 132.

[0049] The Content Generation Module 136 can generate and deliver content to a dialog session that contains one or more content parameters selected from several candidate content parameters. The Content Generation Module 136 can further provide the selected content parameters of the delivered content to the Feedback Engine 140 for use in generating feedback prompts as described herein.

[0050] For example, an input provided in a dialogue session might be "Give me directions to a good café," and the content generation module 136 can determine directions to a specific café by first selecting a "good café" from several available cafés (e.g., based on one or more candidate actions from action module 132 and / or candidate entities from entity module 134) based on one or more factors. Furthermore, the content generation module 136 can determine the directions by selecting them from several candidate route descriptions (e.g., shortest vs. fastest route; include vs. exclude highways) based on one or more factors. Optionally, the content generation module 136 can communicate with one or more external components when determining the specific café and / or the directions.The content generation module 136 can supply the selected content parameters to the feedback engine 140. For example, the content generation module 136 can provide a content parameter specifying the selected cafe and a content parameter specifying one or more parameters of the selected directions (e.g., indicating that this was the "fastest route").

[0051] As another example, the provided input could be "Tell me about my day," and the content generation module 136 could select one or more of the user's calendar entries, local weather for the user, one or more user-specific news items, and / or other content to provide in response to the input. The content generation module 136 could select the content based on various factors, such as the type of content (e.g., "calendar entry," "news"), individual and / or collective size (e.g., duration) of the content, and so on. The content generation module 136 could then deliver the selected content parameters to the feedback engine 140.For example, the content generation module 136 can provide a content parameter indicating that “calendar entries” and “messages” have been provided to the user, and / or a content parameter indicating that the length of an audible presentation of all the content in the dialog session was “2 minutes”.

[0052] The attribute module 138 determines one or more attributes applicable to a user participating in a dialogue session with the automated assistant 120 and provides these attributes to one or more other components of the response content engine 130 for use in generating content to be provided to the user in the dialogue session. The attributes can be used by the other components of the response content engine 130, for example, when determining a specific entity for an action, when determining one or more criteria for the action, and / or when generating output for a dialogue with the one or more users. The attribute module 138 communicates with the attribute database 156, which can store an attribute that is personal to the user and / or applicable to a group of users that includes the user.As described herein, the values ​​of various attributes in attribute database 156 can be influenced based on input provided by one or more users in response to feedback prompts. For example, an attribute in attribute database 156 may contain a value associated with a particular restaurant, indicating the desirability of that restaurant to a specific user. This value can be based on input provided by that user in response to one or more feedback prompts directed at that restaurant, provided to that user in one or more dialog sessions. Attribute database 156 may additionally or alternatively contain other attributes that are personal to a user, but whose values ​​are not necessarily influenced by responses to feedback prompts.For example, such attributes may include the user's current location (e.g., based on GPS or other location data), the user's scheduling constraints (e.g., based on the user's electronic calendar), and / or user attributes based on user activity across multiple internet services.

[0053] As mentioned above, the feedback engine 140 stores various content parameters selected by the response content engine 130 and provided to a user in a dialog session – and generates prompts to request feedback from the user regarding one or more of the content parameters. The feedback engine 140 can include a module for provided content parameters 142, a prompt generation module 144, and a feedback module 146.

[0054] The Provided Content Parameters module 142 stores content parameters of content provided for presentation to a user in dialog sessions with that user, in association with that user, in the Provided Content Parameters database 154. For example, the Content Generation module 136, for an input provided in a dialog session, "What is a good café?", can select a "good café" from several available cafés, provide response outputs indicating the selected "good café", and deliver a content parameter indicating the selected "good café" to the Feedback Engine 140. The Provided Content Parameters module 142 can store the provided content parameter, in association with the user, in the Provided Content Parameters database 154.

[0055] The prompt generation module 144 generates response prompts that request feedback from a user regarding one or more of the content parameters stored in the database for provided content parameters 154 in connection with the user. In some implementations, the prompt generation module 144 also determines when and / or how a response prompt should be provided to the user.For example, the Prompt Generation Module 144 can determine whether to present a feedback prompt to a user as part of a dialog session based on the following criteria: that the dialog session is separate from the dialog session in which one or more content parameters, which are the focus of the feedback prompt, were provided to the user; that user input and / or response output from the dialog session meet one or more criteria; based on verification that the user has responded to a suggestion on which the one or more content parameters are based; and / or based on other criteria. The Prompt Generation Module 144 can work in conjunction with the Response Content Engine 130 to insert generated feedback prompts into specific dialog sessions managed by the Response Content Engine 130.In some implementations, the prompt generation module 144 and / or other components of the feedback engine 140 may be included in the response content engine 130.

[0056] In some implementations, content parameters from several different suggestions presented to the user may be available to the prompt generation module 144 to generate a prompt to be presented to the user in a given dialog session. In some of these implementations, the prompt generation module 144 may select a subset (e.g., one) of these multiple suggestions to present in a given prompt based on one or more criteria.

[0057] The feedback module 146 uses feedback provided by the user in response to the prompt to influence future suggestions and / or other content subsequently provided by the automated assistant 120 in future dialog sessions to the user and / or other users. In some implementations, the feedback module 146 uses one or more instances of feedback to adjust values ​​associated with one or more attributes in the attribute database 156. The adjusted values ​​can be personal to the user and / or applicable to a group of users (e.g., all users). In some implementations, the feedback module 146 can use annotations provided by the natural language processor 122 to determine the influence that one or more instances of feedback have on an associated attribute.For example, the Natural Language Processor 122 can contain a sentiment classifier, provide annotations indicating the sentiment of provided feedback, and the Feedback Module 146 can use the specified sentiment to adjust the value. For example, for a feedback prompt "How did you like Cafe A?" and a user-provided response "It was great," the Feedback Module 146 can receive an annotation indicating that "It was great" is associated with very positive feedback. Based on such an annotation, the Feedback Module 146 can boost the value of an attribute associated with "Cafe A."

[0058] With reference to Fig. 2A, Fig. 2B 1, Fig. 2B2, Fig. 2B3 and Fig. Section 2B4 describes examples of various implementations disclosed herein. Fig. 2A represents an example of a dialogue session in which the automated assistant provides content to a user with content parameters selected from several candidate content parameters. Fig. 2B1- Fig. Figures 2B4 each show a different example of providing one or more feedback prompts to the user in a separate dialog session, with the feedback prompts being based on the selected content parameters.

[0059] Fig. Figure 2A shows a computing device 210 containing one or more microphones and one or more loudspeakers, and shows examples of dialogue sessions that can take place via the one or more microphones and the one or more loudspeakers between a user 101 of the computing device 210 and the automated assistant 120 according to the implementations described herein. One or more aspects of the automated assistant 120 can be implemented on the computing device 210 and / or on one or more computing devices that are in network communication with the computing device 210.

[0060] In Fig. In step 2A, the user provides the natural language input 280A, "Can you order dinner for 6:00 PM?", to initiate a dialogue session between the user and the automated assistant 120. In response to the natural language input 280A, the automated assistant 120 provides a natural language output 282A, "Sure, what would you like?". The user then provides a natural language input 280B, indicating that they would like Mexican food. The automated assistant 120 then provides a natural language output 282B, asking the user if they would like the automated assistant to select a specific restaurant, to which the user responds with a confirming natural language input 280C.

[0061] The automated assistant 120 then delivers a natural language output 282C, which is a suggestion. The natural language output suggestion 282C includes a specific restaurant (Café Lupe) selected from several candidate restaurants serving "Mexican cuisine," and further includes a specific dish ("Burrito") selected from a variety of candidate dishes available at that particular restaurant.

[0062] The user then provides a natural language input 280D, instructing the automated assistant 120 to order a burrito and fries from Cafe Lupe. The automated assistant 120 then provides a natural language output 282D to confirm that the user's request from the input 280D has been fulfilled by the automated assistant 120 (optionally using one or more additional external components).

[0063] In the dialogue session of Fig. 2A, the automated assistant 120 selected the location "Cafe Lupe" and the dish type "Burrito" from several candidate options and presented this selection to the user as part of the dialogue session. Furthermore, the automated assistant 120 can determine that the user accepted the suggestion because the user did so in the same dialogue session (by ordering the burritos from Cafe Lupe). Based on the recommendation of the location and dish type, and optionally based on the determination that the user accepted the suggested location and dish type, the automated assistant 120 can store content parameters specifying the suggested location and dish type. For example, the automated assistant 120 can store a record of these selected and presented content parameters in the database for provided content parameters 154 in conjunction with the user.

[0064] Fig. 2B 1 provides an example of providing a feedback prompt to the user based on the content parameter "Cafe Lupe", which is a response to the dialog session of Fig. 2A is stored. Fig. 2B 1, the user provides the natural language input 280A1 "What does my daily schedule look like for today?" to initiate a dialogue session between the user and the automated assistant 120. The dialogue session of Fig. 2B 1 is separate from that of Fig. 2A. For example, based on the lapse of at least one threshold period since the dialogue session of Fig. 2A and / or based on other criteria, it will be determined that the dialogue session is by Fig. 2B1 is a separate dialogue session.

[0065] The automated assistant 120 responds to natural language input 280A1 with the responding natural language output 282A1, which includes a summary of the user's calendar along with a local forecast and a traffic report. Even though the input 280A1 and the output 282A1 are not related to the content of the dialogue session of Fig. If 2A is related, the automated assistant 120 then provides a feedback request 282B 1, which requests feedback on the content parameter "Cafe Lupe". As described herein, the automated assistant 120 can issue the feedback request 282B 1 based on various criteria, such as whether the dialog session is from Fig. 3B1 separate from that of Fig. 3A is that the input 380A1 and / or the output 382A1 meets one or more criteria, such as being considered a “routine”, and / or provide other criteria.

[0066] The user responds to the feedback prompt 282B1 with a positive natural language input 280B1, and the automated assistant responds with an affirmative natural language output 282C1. The automated assistant 120 can use the positive natural language input 280B1 to positively influence a value associated with the content parameter "Cafe Lupe." For example, the automated assistant 120 can adjust the value to increase the likelihood that "Cafe Lupe" and / or restaurants similar to Cafe Lupe will be provided in future dialogue sessions with the user and / or in future dialogue sessions with other users.

[0067] Fig. 2B2 provides an example of providing a feedback prompt to the user based on the content parameter "Burrito", which is a response to the dialog session of Fig. 2A is saved. The dialog session of Fig. 2B2 can be one that replaces or is an addition to the one that is Fig. 2B1 takes place. Fig. 2B2, the user provides the natural language input 280A2 "Good morning" to initiate a dialogue session between the user and the automated assistant 120. The dialogue session of Fig. 2B2 is from that of Fig. 2B separated. For example, based on the lapse of at least one threshold time period since the dialogue session of Fig. 2A and / or based on other criteria, it will be determined that the dialogue session is by Fig. 2B1 is a separate dialogue session.

[0068] The automated assistant 120 responds to the natural language input 280A2 with the responding natural language output 282A2, which contains the response content "Good morning John". Although the input 280A2 does not correspond to the content of the dialogue session of Fig. In relation to 2A, the natural language output 282A2 also includes feedback such as "How did you like the burrito from Cafe Lupe last night?". As described herein, the automated assistant 120 can provide the feedback prompt contained in output 282A2 based on various criteria.

[0069] The user responds to output 282A2 with a positive natural language input 280B2. The automated assistant responds with natural language output 282B2, which includes the affirmative word "beautiful," and a suggestion for another popular dish at Cafe Lupe. Automated assistant 120 can use the positive natural language input 280B2 to positively influence a value associated with the content parameter "burrito." For example, automated assistant 120 can adjust the value to increase the likelihood that "burrito" will be recommended as a dish at Cafe Lupe and / or other restaurants in future dialogue sessions with the user and / or in future dialogue sessions with other users.

[0070] Fig. 2B3 provides another example of providing a feedback prompt to the user based on the content parameter "Burrito", which is a response to the dialog session of Fig. 2A is saved. The dialog session of Fig. 2B3 can be one that replaces or is in addition to that of Fig. 2B1 and / or Fig. 2B2 takes place. Fig. 2B3 The user provides the natural language input 280A3 "Please play some music" to initiate a dialogue session between the user and the automated assistant 120. The dialogue session of Fig. 2B2 is from that of Fig. 2A separated.

[0071] The automated assistant 120 responds to the natural language input 280A3 with the response output 282A3, which contains a song (indicated by musical notes) as the answer. Although the input 280A3 does not correspond to the content of the dialogue session of Fig. Related to 2A, output 282A3 also contains a feedback prompt, "By the way, how did you like the burrito from Cafe Lupe last night?". This feedback prompt can be provided after the song, or a portion of it, has finished playing. As described herein, the automated assistant 120 can provide the feedback prompt contained in output 282A3 based on various criteria.

[0072] The user responds to output 282A3 with a positive natural language input 280B3. The automated assistant responds with output 282B3, which contains the affirmative words "Glad to hear it," and then continues playing the response song or an additional song. The automated assistant 120 can use the positive natural language input 280B3 to positively influence a value associated with the content parameter "Burrito."

[0073] Fig. 2B4 provides another example of providing a feedback request to the user based on the content parameter "Cafe Lupe", which is a response to the dialog session of Fig. 2A is saved. The dialog session of Fig. 2B4 can be one that replaces or is in addition to that of Fig. 2B 1, Fig. 2B2 and / or Fig. 2B3 takes place.

[0074] Fig. 2B4 illustrates another client device 310 of user 101. Fig. 2A and a display screen 320 of the client device 310. The client device 310 can contain and / or communicate with the automated assistant 120 and / or another instance thereof (which has access to entries of the user 101 in the content parameter database 154). The display screen 340 contains a response interface element 388, which the user can select to generate user interface input via a virtual keyboard, and a voice response interface element 389, which the user can select to generate user interface input via a microphone. In some implementations, the user can generate user interface input via the microphone without selecting the voice response interface element 389.For example, during the dialogue session, active monitoring for audible user interface input via the microphone can take place to eliminate the need for the user to select the voice response interface element 389. In some of these and / or other implementations, the voice response interface element 389 can be omitted. Furthermore, in some implementations, the response interface element 388 can be additionally and / or alternatively omitted (e.g., the user can provide only audible user interface input). The display screen 340 also contains system interface elements 381, 382, ​​and 383, which the user can interact with to cause the client device 310 to perform one or more actions.

[0075] In Fig. 2B4 The user provides the natural language input 280A4 "What will my day look like tomorrow?" to initiate a dialogue session between the user and the automated assistant 120. The dialogue session of Fig. 2B2 is from that of Fig. 2A separate. In some implementations, the automated assistant 120 can determine that it is a separate dialog session based on the fact that it is running over a separate client device.

[0076] The automated assistant 120 responds to the natural language input 280A4 with the response output 282A4, which contains a summary of the user's calendar. Although the input 280A4 and the output 282A4 do not correspond to the content of the dialog session of Fig. When 2A is in relation to the user, the automated assistant then provides a feedback prompt 282B4, "By the way, did you like Cafe Lupe?" The user responds to the feedback prompt 282B4 with a positive natural language input 280B4. In some implementations, the natural language input 280B4 can be free-form. In some other implementations, the automated assistant 120 can present several options for the user to choose from in the dialog. For example, the automated assistant 120, in combination with the feedback prompt 282B4, can provide an interface that includes several options such as "Yes," "No," and "It was OK"—and the user can select one of the options to provide a corresponding response input.

[0077] Fig. 2B 1- Fig. 2B4 provides examples where the user gives positive feedback and the corresponding content parameters are upgraded. However, it is understood that the user can alternatively provide negative feedback, which downgrades the corresponding content parameters.

[0078] With reference to Fig. 3A and Fig. Section 3B describes additional examples of various implementations disclosed here. Fig. 3A presents an example of a dialog session in which the automated assistant provides content to a user with a content parameter selected from several candidate content parameters. Fig. Figure 3B shows an example of providing feedback prompts to the user in a separate dialog session, where the feedback prompts are based on the selected content parameters.

[0079] Fig. 3A shows the same client device 310 as the one in Fig. 2B4 shown. In Fig. In step 3A, a dialogue session takes place between the user ("You"), an additional user ("Tom"), and the automated assistant 120 ("Automated Assistant"). The user provides a natural language input 380A1, "Coffee in the morning?", addressed to the additional user. The additional user provides the responding natural language input 381A1, "Sure". The user then calls the automated assistant 120 into the dialogue session by including "@AutomatedAssistant" in the input 380A2 and asks the automated assistant 120: "Find a good cafe!". In response, the automated assistant 120 provides output 382A1 containing the suggestion "Hypothetical Roastery", which is highly rated and close to both the user and the additional user.Output 382A1 can be made available for presentation to both the user (via client device 310) and the additional user (via a corresponding client device). The automated assistant 120 selects the location "Hypothetical Roastery" from several candidate options. Furthermore, the automated assistant 120 can determine, based on other signals associated with the user (e.g., the user issuing a navigation request regarding the "Hypothetical Roastery," user location data indicating a visit to the "Hypothetical Roastery") and / or other signals, that the user has responded to the suggestion. Based on the location recommendation and, optionally, on the determination that the user has responded to the suggested location, the automated assistant 120 can store content parameters in conjunction with the user that specify the suggested location.For example, the automated assistant 120 can store a record of the selected and provided content parameters in the database for provided content parameters 154 in conjunction with the user. The automated assistant 120 can additionally or alternatively store content parameters in conjunction with the additional user that specify the suggested location (optionally after determining that the additional user has responded to the suggested location).

[0080] Fig. 3B shows an example of providing feedback prompts based on the selected content parameter of Fig. 3A based, for user 101 in a dialog session that differs from that of Fig. 3A is separated. In Fig. 3B, the user 101 provides a natural language input 380B 1 "What is the journey home like?" to the separate computing device 210. The dialogue session of Fig. 3B is from that of Fig. 3A separated. For example, the dialogue session can be from Fig. 3B is determined to be a separate dialogue session based on the fact that it takes place via the separate computing device 210.

[0081] The automated assistant 120 responds to the natural language input 380B1 with the responding natural language output 382B1, which contains a summary of the current traffic conditions. Even though the input 380B1 and the output 382B1 are not related to the content of the dialogue session of Fig. If 3A is related, the automated assistant 120 then provides a feedback request 382B2, which requests feedback on the content parameter "Hypothetical Roastery". As described herein, the automated assistant 120 can issue the feedback request 382B2 based on various criteria, such as whether the dialog session is from Fig. 3B separate from that of Fig. 3A is that the input 380B 1 and / or the output 382B1 meets one or more criteria, such as being considered a “routine”, and / or provide other criteria.

[0082] The user responds to output 382B1 with negative natural language input 380B2. The automated assistant provides a further feedback prompt 382B3, requesting feedback on whether there was anything specific the user disliked about the "hypothetical roastery." The user responds to output 382B3 with another natural language input 380B3, specifying that the "hypothetical roastery" was too crowded. The automated assistant 120 can use negative natural language input 380B2 to negatively influence a value associated with the content parameter "hypothetical roastery." The automated assistant 120 can additionally or alternatively use natural language input 380B3 to influence a value associated with a content parameter indicating a visit level.For example, the automated assistant 120 can adjust a value to reduce the likelihood that restaurants associated with "large crowds" at one or more times will be recommended at those times in future dialogue sessions with the user and / or in future dialogue sessions with other users.

[0083] Fig. Figure 3B presents an example with user 101 of client device 310. However, a separate instance of the automated assistant 120 can additionally and / or alternatively, based on the selected content parameter stored in the database for provided content parameters 154 in connection with the additional user, send feedback requests to the additional user ("Tom"). Fig. Provide 3A via one of its client devices.

[0084] Although the examples provided in various figures focus on location suggestions, the techniques described herein can be implemented for other types of suggestions and / or other content. For example, in response to a user input of "Play some bluegrass," Automated Assistant 120 can select a bluegrass album A to play as a reply. Automated Assistant 120 can then provide a prompt such as "How did you like bluegrass album A?" in a future dialogue session. As another example, in response to a user input of "Breaking News," Automated Assistant 120 can select breaking news from source A to play as a reply.The automated assistant 120 can then, in a future dialog, provide a prompt such as, "Would you like Source A for the news, or would you prefer a different source?" As another example, in response to a user input of "Latest news," the automated assistant 120 can select five news articles to provide. The automated assistant 120 can then, in a future dialog session, provide a prompt such as, "Were you satisfied with the number of news articles previously provided, or would you prefer more or fewer?" As yet another example, in response to a user input of "Navigate to a cafe," the automated assistant 120 can select the nearest cafe and provide directions to it.The automated assistant 120 can then provide a prompt in a dialogue session the following day, such as "Yesterday I gave you directions to the nearest cafe. In the future, would you prefer a slightly higher-rated cafe?"

[0085] Fig. Figure 4 is a flowchart illustrating an exemplary procedure 400 according to implementations disclosed herein. For simplicity, the operations of the flowchart are described with reference to a system that performs the operations. This system may include various components from different computer systems, such as one or more components of the automated assistant 120. Although operations of procedure 400 are shown in a particular order, this is not intended to be restrictive. One or more operations may be rearranged, omitted, or added.

[0086] In block 452, the system receives natural language input based on user interface input provided by a user during a dialog session.

[0087] In block 454, the system generates content in response to natural language input, which contains at least one content parameter selected from several candidate content parameters.

[0088] In block 456, the system provides the content for presentation to the user as part of the dialog session and saves the selected content parameter of the content.

[0089] In block 458, the system identifies an additional dialog session that includes the user. In some implementations, block 458 may contain a block 460 in which the system generates and provides additional content for presentation to the user as part of the additional dialog session.

[0090] In block 462, the system provides the user with a prompt as part of the additional dialog session, requesting feedback on the saved selected content parameter.

[0091] In block 464, the system receives an additional input in response to the prompt, based on user interface input provided by the user during the additional dialog session.

[0092] In block 466, the system uses the additional input to influence a value that is stored in connection with the selected content parameter.

[0093] Fig. Figure 5 is a block diagram of an exemplary computing device 510, which can optionally be used to perform one or more aspects of the techniques described herein. In some implementations, a client computing device, an automated assistant 120, and / or one or more other components may include one or more components of the exemplary computing device 510.

[0094] The computing device 510 typically contains at least one processor 514, which communicates with a number of peripheral devices via the bus subsystem 512. These peripheral devices may include a storage subsystem 524, which may contain, for example, a memory subsystem 525 and a file storage subsystem 526, user interface output devices 520, user interface input devices 522, and a network interface subsystem 516. The input and output devices enable user interaction with the computing device 510. The network interface subsystem 516 provides an interface to external networks and is coupled with corresponding interface devices in other computing devices.

[0095] The user interface input devices 522 may include a keyboard, pointing devices such as a mouse, trackball, touchpad or graphics tablet, a scanner, a touchscreen integrated into the display, audio input devices such as speech recognition systems, microphones and / or other types of input devices. In general, the use of the term “input device” is intended to encompass all possible types of devices and means of inputting information into the computing device 510 or into a communications network.

[0096] The user interface output devices 520 may include a display subsystem, a printer, a fax device, or non-visual displays such as audio output devices. The display subsystem may include a cathode ray tube (CRT), a flat-panel display device such as a liquid crystal display (LCD), a projection device, or any other mechanism for producing a visible image. The display subsystem may also provide a non-visual display, such as via audio output devices. In general, the use of the term "output device" is intended to encompass all possible types of devices and means of outputting information from the computing device 510 to the user or to another machine or computing device.

[0097] The 524 storage subsystem stores program and data constructs that provide the functionality of some or all of the modules described here. For example, the 524 storage subsystem can contain the logic to execute selected aspects of the procedure of Fig. 4 to execute.

[0098] These software modules are generally executed by the 514 processor alone or in combination with other processors. The 525 memory, used in the file storage subsystem 524, can include several memories, including a main random-access memory (main RAM) 530 for storing instructions and data during program execution, and a read-only memory (ROM) 532 in which fixed instructions are stored. The file storage subsystem 526 can provide persistent memory for program and data files and can include a hard disk drive, a floppy disk drive along with associated removable media, a CD-ROM drive, an optical drive, or removable media cartridges.The modules that implement the functionality of certain implementations can be stored by the file storage subsystem 526 in the storage storage subsystem 524 or in other machines that can be accessed by the processor(s) 514.

[0099] The Bus Subsystem 512 provides a mechanism to enable the various components and subsystems of the Computing Device 510 to communicate with each other as intended. Although the Bus Subsystem 512 is schematically shown as a single bus, alternative implementations of the Bus Subsystem may use multiple buses.

[0100] The computing device 510 can be of various types, including a workstation, server, computer cluster, blade server, server farm, or other data processing system or device. Due to the constantly evolving nature of computers and networks, the description of the in Fig. The computing device 510 shown in Figure 5 is intended only as a specific example for the purpose of illustrating some implementations. Many other configurations of the computing device 510 are possible, with more or fewer components than those shown. Fig. 5 depicted calculating device.

[0101] In situations where certain implementations discussed herein may collect or use personal information about users (e.g., user data extracted from other electronic communications, information about a user's social network, a user's location, a user's time, a user's biometric information and activities, and demographic information), users will be provided with one or more options to control whether information is collected, whether the personal information is stored, whether the personal information is used, and how the information about the user is collected, stored, and used. That is to say, the systems and procedures discussed herein will collect, store, and / or use personal user information only if they receive explicit authorization from the relevant users to do so.For example, a user is given control over whether programs or functions collect user information about that specific user or other users relevant to the program or feature. Each user for whom personal information is to be collected is given one or more options to control the information collection relevant to that user, granting permission or authorization as to whether the information is collected and which parts of the information are collected. For example, users can be provided with one or more such control options via a communication network. Furthermore, certain data can be processed in one or more ways before being stored or used, so that personally identifiable information is removed.As one example, a user's identity can be treated in such a way that no personally identifiable information can be determined. As another example, a user's geographic location can be generalized to a larger area, so that the user's specific location cannot be determined.

[0102] According to the implementations described in this disclosure, methods, devices, and computer-readable media are provided that relate to requesting feedback from a user regarding one or more content parameters of a suggestion or other content provided by the automated assistant. The user's feedback can be used to influence future suggestions and / or other content subsequently provided by the automated assistant in future dialogue sessions for the user and / or other users.In some implementations, content is provided to a user by an automated assistant in a dialog session between the user and the automated assistant. In a subsequent dialog session, the automated assistant then provides a prompt requesting user feedback on the provided content. In some of these implementations, the prompt in the subsequent dialog session is provided after user input and / or output from the automated assistant that is unrelated to the content provided in the previous dialog session.

[0103] All parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and the actual parameters, dimensions, materials, and / or configurations will depend on the specific application(s) for which the teachings are used. It is therefore understood that the foregoing implementations are only exemplary and that implementations within the scope of the appended claims may be carried out differently than specifically described.

Claims

[1] A method that is implemented by one or more processors and includes: as part of a dialogue session between a user and an automated assistant implemented by one or more of the processors: Receiving natural language input based on user interface input provided by the user via a user interface input device; and Providing response content in response to natural language input as a response from the automatic assistant to the natural language input, where the content is provided to the user for presentation via a user interface output device and wherein the content contains at least one content parameter which is selected by the automated assistant from several candidate content parameters; as part of an additional dialogue session between the user and the automated assistant, which is temporally separate from the dialogue session: Providing a prompt that requests feedback regarding the selected content parameters; wherein the prompt to present is provided to the user via the user interface output device or an additional user interface output device, and where the prompt is generated based on the fact that the content parameter was previously selected by the automated assistant and previously made available for presentation to the user as part of the dialog session, in order to request feedback regarding the content parameter, based on providing the prompt, preemptively activating a microphone designed to process the user interface inputs provided via the microphone, wherein the user interface input device or the additional user interface input device includes the microphone, and Receiving additional input in response to the prompt, wherein the additional input is based on additional user interface input provided by the user via the microphone; and Using the additional inputs to influence a value stored in association with the content parameter, wherein the value stored in association with the content parameter influences the future provision of further content containing the content parameter. [2] Method according to claim 1, wherein the user initiates the additional dialogue session with additional natural language inputs that are unrelated to the content of the previous dialogue session. [3] The method of claim 2, further comprising as part of the additional dialogue session: Providing additional dialogue session output in response to additional natural language input that is unrelated to the content of the previous dialogue session; the provision of the input prompt occurs after the provision of the additional dialog session outputs. [4] The method of claim 3, wherein the method further comprises: Determine that one or more criteria are met by the additional natural language inputs and / or the additional dialogue session outputs; where the provision of the prompt is further based on determining that the criteria are met. [5] Method according to claim 4, wherein the criteria include at least one of the following criteria: that the additional dialog session outputs are of a certain semantic type, and / or that at least one N-gram of a set of criteria N-grams occurs in the user input prompt. [6] A method according to any of the preceding claims, wherein the content is a proposal to which the user should respond in the future, and the method further comprises: Determine that the user has accepted the suggestion after the content containing the content parameter has been provided; where the provision of the prompt is further based on determining that the user has responded to the suggestion. [7] Method according to any of the preceding claims, wherein the user initiated the additional dialog session and wherein the provision of the prompt depends on the user having initiated the additional dialog session. [8] A method according to any of the preceding claims, further comprising as part of the additional dialogue session: Before providing the prompt, providing additional dialog session outputs; the provision of the input prompt occurs after the provision of the additional dialog session outputs. [9] A method according to any of the preceding claims, further comprising: Identifying an additional content parameter that is provided to the user as part of an additional prior dialogue session between the user and the automated assistant; and Determine, based on one or more criteria, to provide the prompt based on the content parameter instead of an alternative prompt based on the additional content parameter. [10] Method according to claim 9, wherein one or more criteria comprise a corresponding temporal proximity of the provision of the content parameter and the provision of the additional content parameter. [11] Method according to claim 9 or 10, wherein one or more criteria additionally or alternatively comprise semantic types associated with the content parameter and the additional content parameter. [12] Method according to any of the preceding claims, wherein the content contains a suggestion regarding a specific physical location and a specific article consumable at the specific location and wherein the content parameter identifies the specific article. [13] Method according to any of the preceding claims, wherein the content contains a proposal regarding a specific physical location and the content parameter identifies a category to which the specific physical location belongs. [14] Method according to one of the preceding claims, wherein both the inputs and the additional inputs are generated via the user interface input device and wherein both the content and the feedback request for presentation are provided via the user interface output device. [15] Method according to claim 14, wherein the user interface input device includes a microphone of a single device and the user interface output device includes a loudspeaker of the single device. [16] Method according to any one of the preceding claims, wherein the inputs are generated via the user interface input device of a first computing device, the content for the presentation is provided via the user interface output device of the first computing device and The prompt for the presentation is provided via the additional user interface output device of an additional computing device. [17] A method according to any of the preceding claims, wherein the at least one content parameter contained in the content is stored on a computer-readable medium, comprising: Identifying the stored content parameter from the computer-readable medium; and where providing a prompt requesting feedback regarding the content parameters includes providing a prompt requesting feedback regarding the stored content parameter. [18] Computer program containing machine-readable instructions which, when executed by a computing device, cause it to perform the method according to any one of claims 1 to 17. [19] Client device containing the following: at least one microphone; at least one loudspeaker; a network interface; one or more processors designed to perform the method according to any one of claims 1 to 17.

Images