Generate automated assistant responses and / or actions directly from conversation history and resources

Abstract

The present disclosure relates to generating automated assistant responses and / or actions directly from conversation history and resources. A single neural network model is trained and / or utilized to generate corresponding automated assistant natural language responses and / or corresponding automated assistant actions at each of multiple assistant turns of a conversation session between a user and an automated assistant. For example, at a given assistant turn of a conversation session, a corresponding natural language response and a corresponding action may be generated jointly and directly based on outputs generated using a single neural network model. The corresponding response and / or corresponding action may be generated based on processing a conversation history and multiple discrete resources using a neural network model. For example, a neural network model may be used to generate responses and / or actions on a token-by-token basis.

Description

Technical Field

[0001] The present disclosure relates to generating automated assistant responses and / or actions directly from conversation history and resources. Background Art

[0002] People can participate in human-computer dialogues using interactive software applications referred to herein as "automated assistants" (also referred to as "digital agents," "interactive personal assistants," "intelligent personal assistants," "assistant applications," "conversational agents," etc.). For example, humans (who may be referred to as "users" when they interact with automated assistants) can provide commands and / or requests to automated assistants using verbal natural language input (i.e., utterances) and / or by providing textual (e.g., typed) natural language input, which in some cases can be converted to text and then processed. The automated assistant responds to the request by providing responsive user interface output and / or taking responsive actions. The responsive user interface output may include natural language output, which may optionally be converted to synthesized speech and rendered. The responsive action may include, for example, controlling a peripheral device (e.g., a smart light, a smart appliance, etc.) and / or interacting with another system to perform a task (e.g., requesting a ride share, booking a restaurant, etc.).

[0003] Automated assistants typically rely on a pipeline of multiple discrete components when interpreting and responding to user requests. For example, a natural language understanding component can be used to process the text of a request (e.g., text converted from a spoken utterance) to generate a symbolic representation or belief state, i.e., a semantic representation of the text. For example, a belief state can include an intent corresponding to the text and optionally parameters for the intent (e.g., time slot values). Separate implementation components can then use the belief state to reason about external knowledge sources, the results of which are used independently for action prediction tasks and response generation tasks. However, component-dependent pipelines can make automated assistants difficult to scale. Moreover, each individual component must be trained separately, which can require a lot of engineering effort and require extensive use of computer resources during training. In addition, global enhancement signals (i.e., those that depend on the overall results of a conversation turn or a complete conversation) may be problematic and / or impossible to use effectively to train a single component. Summary of the invention

[0004] Implementations disclosed herein are directed to training and / or utilizing a single neural network model to generate a corresponding automated assistant natural language response and / or a corresponding automated assistant action at each of a plurality of assistant turns of a conversation session between a user and an automated assistant. For example, at a given assistant turn of a conversation session, a corresponding natural language response and a corresponding action may be jointly generated, and may be generated directly based on output generated using a single neural network model.

[0005] The corresponding response and / or the corresponding action can be generated based on processing the conversation history (i.e., user utterances, previous automated assistant responses, and / or previous automated assistant actions) and multiple discrete resources (e.g., knowledge graph entries, search results, and / or search result documents) using a neural network model. For example, the neural network model can be used to generate responses and / or actions on a token-by-token basis, where each token is a word, a word fragment, or a character. For example, at each iteration of multiple iterations, a corresponding probability distribution over a vocabulary of tokens can be generated using a neural network model, and the token with the highest probability is selected in that iteration. Thus, over multiple iterations (e.g., until the "end" token has the highest probability), a sequence of the highest probability tokens is selected, and the sequence is a natural language response and / or action. The natural language response can be rendered as an automated assistant response (e.g., as an audible output based on text-to-spoken utterance processing), and any action can be initiated by the automated assistant (e.g., transmitted to a remote device to cause the action to be performed).

[0006] In these and other ways, automated assistant responses and actions can be generated without using a separate natural language component and then a separate fulfillment component to generate belief states. By training and utilizing a single neural network model according to the implementations disclosed herein, automated assistant response and action generation can be performed across multiple domains and for a variety of conversations, thereby enabling efficient scaling of automated assistant functionality across domains and / or conversations. In addition, the global enhancement signal can be used to efficiently train a single neural network model, thereby alleviating the need for resource- and labor-intensive annotation of belief states when training a separate natural language component and / or a separate fulfillment component.

[0007] In some implementations, a single neural network model includes a transformer encoder and a transformer decoder. In some versions of these implementations, the transformer encoder is used to process a cascade of dialogue histories and a cascade of discrete resources to generate an encoded representation containing an encoded dialogue representation (of the dialogue history) and an encoded resource representation (of the discrete resources). In some other versions of those implementations, the transformer encoder is used to process the dialogue history to generate the encoded dialogue representation, and the encoded resource representation is based on separate processing of the discrete resources. The separate processing may include, for example, generating an average word embedding and / or other initial representation for each discrete resource. The separate processing may additionally or alternatively include attention fusion, wherein the initial representation of the discrete resource is modified based on the corresponding probability of the discrete resource determined using a sparse attention mechanism and the encoded dialogue representation (e.g., by multiplication). As described herein, when a sparse attention mechanism is used in various implementations, utilizing the sparse attention mechanism can result in effective natural language response and / or action prediction using a single neural network model. For example, the sparse attention mechanism can enable a large number (e.g., thousands) of different resources to be considered when operating effectively in natural language response and / or action prediction. Additionally or alternatively, utilization of a sparse attention mechanism can alleviate the overfitting problem that would otherwise arise due to a limited number of training examples and / or the spread of scarce tokens in the utilized discrete resources.

[0008] In various implementations, the converter decoder is used to generate natural language responses and / or actions token by token by having the converter decoder's attention head process both the encoded dialogue representation and the encoded resource representation. In some of those various implementations, processing both the encoded dialogue representation and the encoded resource representation can enable the decoder to be used in effective response and / or action generation taking into account the dialogue history and related (as indicated by the encoded resource representation attention) discrete resources. As an example, in a first iteration, the "start" token can be processed using a converter decoder that processes both representations to generate a first probability distribution over a token vocabulary. The highest probability token can be selected as the first token of the natural language response and / or action. The converter decoder that processes both representations can then be used to process the start token and the highest probability token to generate a second probability distribution over a token vocabulary. The highest probability token can be selected as the second token of the natural language response and / or action. The process can continue, selecting a new token in each iteration and adding it to the input processing of the next iteration until the "end" token is selected. The resulting token sequence can then be used as a natural language response and / or action performed by the automated assistant.

[0009] Some implementations disclosed herein are directed to a transformer encoder, a transformer decoder, and optionally (when utilized) a sparse attention mechanism for training a single neural network model utilized in natural language response and / or action generation. Training instances may be utilized, each training instance comprising a ground-truth prior conversation history (i.e., user utterances, automated assistant responses, and / or automated assistant actions) and a ground-truth next automated assistant response and / or automated assistant action.

[0010] In some of these implementations, and during at least a portion of the training, only a subset of the available discrete resources is processed with the training instance (while all available discrete resources can be optionally processed at runtime). For example, a positive subset of the discrete resources of the training instance can be selected based on the item that matches (soft and / or exact match) with the ground truth next automatic response and / or action of the training instance. In addition, in order to increase the training speed (by including negative discrete resources with weaker labels), a negative subset of the remaining discrete resources can be randomly selected for the training instance. In general, the positive subset and the negative subset still constitute a subset of the available discrete resources. For example, they can account for less than 50%, less than 10%, or less than 5% of the available discrete resources. Considering only a subset of discrete resources in each iteration can also increase the training speed, and selecting a positive subset through remote supervision can achieve effective training, while selecting a negative subset can further increase the training speed. Moreover, and as described herein, in an implementation utilizing a sparse attention mechanism, the utilization of weak supervisory signals from remote supervision can guide the training of the sparse attention mechanism.

[0011] In some implementations in which a sparse attention mechanism is trained, the sparse attention mechanism may not be trained for the first M steps of training (e.g., 10,000, 20,000, or other number) and / or the first M durations of training (e.g., 10 hours, 15 hours, or other time). For example, during those steps, the converter encoder and / or converter decoder weights may be updated without updating the weights of the sparse attention mechanism. For the first M steps and / or M durations, positively labeled electronic resources determined from weak labels based on remote supervision can be used as electronic resources, and the decoder processes the representation of these electronic resources. These first M training steps will warm start the model and train other parts of the model except for attention. Optionally, during the first M training steps and / or during the M durations, negatively labeled electronic resources determined from weak labels based on remote supervision are not utilized. For the next N (e.g., 20,000 or other number) steps and / or N durations, the sparse attention mechanism is trained. The sparse attention mechanism can be trained based on weak supervision signals (e.g., comparing probabilities predicted using the sparse attention mechanism with weak labels based on remote supervision) and / or based on responses and actions predicted by the decoder for ground truth responses and / or actions. In addition, for the next N steps and / or N durations, sparse attention to the KB is inserted between the remote supervision and the probabilities (e.g., S-shaped probabilities) generated from the model using the sparse attention mechanism. For example, when determining the electronic resource-based attention to the decoder, for positively labeled electronic resources, it can be based on the insertion between the sparse S-shaped probability of the resource and the positive (e.g., "1") probability of the electronic resource inferred from the remote supervision. In some implementations, the weight on the remote supervision gradually decays (e.g., from 1.0 to 0.0) over N steps. This enables the sparse attention mechanism to be trained over N steps while guiding the electronic resource-based attention for the decoder during the training of the sparse attention mechanism (while reducing the degree of guidance as the sparse attention mechanism is further trained). For the last O (e.g., 10,000 or other number) steps and / or O duration, the sparse attention mechanism will continue to be trained. However, in the last O steps, only the sparse attention mechanism is used to determine the decoder's electronic resource-based attention without considering distant supervision (or only using distant supervision very little).

[0012] The above is merely provided as an overview of some implementations disclosed herein. These and other implementations are further described in detail below.

[0013] Various implementations disclosed herein may include a transient or non-transitory computer-readable storage medium storing instructions executable by one or more processors (e.g., one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or one or more tensor processing units (TPUs)) to perform a method, such as one or more methods described herein. Another implementation may include a system of one or more computers including one or more processors operable to execute stored instructions to perform a method, such as one or more of the methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram of an example environment in which implementations disclosed herein may be implemented.

[0015] Figure 2A Shows Figure 1 The implementation of the neural network model and Figure 1 How the response and action generation engine leverages it to generate natural language responses and optionally actions.

[0016] Figure 2B Shows Figure 1 Another implementation of the neural network model and Figure 1 How the response and action generation engine leverages it to generate natural language responses and optionally actions.

[0017] Figure 3 is a flow chart illustrating an example method for generating natural language responses and optionally actions directly from a conversation history and discrete resources.

[0018] Figure 4 is another flow chart illustrating an example method for generating natural language responses, and optionally actions, directly from a conversation history and discrete resources.

[0019] Figure 5 is a flow chart illustrating an example method for training a single neural network model that can be used to generate natural language responses and, optionally, actions directly from conversation history and discrete resources.

[0020] Figure 6 An example architecture for a computing device is shown. DETAILED DESCRIPTION

[0021] Before describing the accompanying drawings, some specific examples of various implementations are described. The implementation disclosed herein presents a single neural network model that processes a conversation history and an external knowledge source (which includes multiple discrete resources, such as knowledge triples) as input, and jointly generates a text response and the operation to be performed by the system (if any) as output. The action can also be represented in text / token form. For example, the smart thermostat temperature reduction adjustment action can take the form of "adjust (action = reduce, quantity = 2)", where "adjust" defines the intention of the action, "reduce" defines the value of the increase / decrease parameter, and "2" defines the value of the degree used to adjust. As another example, the hotel reservation action can take the form of "hotel reservation (night = 2, person = 4, day = Friday)", where "hotel reservation" defines the intention of the action, "2" defines the value of the night number parameter, "4" defines the value of the number of people parameter, and "Friday" defines the value of the check-in date parameter.

[0022] By training according to the implementation disclosed herein, the neural network model effectively learns to use weak supervisory signals from text generation and action prediction tasks to reason about the provided knowledge source, thereby eliminating the need for belief state annotations. A variety of different strategies can be optionally used to effectively utilize external knowledge sources and neural network models. For example, remote supervision can be used as an additional source of (weak) supervisory signals to help the model training enable the selection (and / or processing) of relevant discrete resources from large knowledge sources. For example, remote supervision can be performed by making the model distinguish between weakly labeled positive examples and randomly (e.g., pseudo-random or truly random) sampled negative examples. As another example, a sparse attention mechanism on an external knowledge source can be additionally or alternatively utilized to prevent overfitting training instances. As yet another example, the output (e.g., probability distribution) of the generative model can be additionally or alternatively modified to enable the option of copying tokens from conversation history and / or discrete resources, rather than always generating them. This is due to the recognition that the desired output tokens can contain entries from the knowledge source, which are typically rare tokens, and if not modified, it may be difficult for the model to predict them directly. As yet another example, in various implementations (e.g., where a neural network model is utilized to generate text responses and actions), scheduled sampling may be utilized during the training process in place of (or in addition to) teacher-forced training.

[0023] In some implementations, a dataset containing actual and / or hand-crafted multi-turn conversations between users and automated assistants can be used to generate training instances for training a neural network model. A non-limiting example is the MultiWOZ dataset, which contains 10,000 multi-turn conversations between users and guides. Along with the conversations, the dataset also contains belief state and dialogue act (or semantic parse) annotations. Only belief state annotations corresponding to action predictions are used (when generating ground truth automated assistant actions), while other belief states are removed from the dataset. Dialogue act annotations are not used.

[0024] Task-oriented dialogue problems can be formulated as taking a conversation history and the associated knowledge source / knowledge base (KB) as input and generating a system action and a next-turn text response from the assistant as output. For example, the conversation history may contain a single-turn user utterance "Find me a cheap Italian restaurant in San Francisco", and one of the possible next-turn responses from the assistant may be "What about great Italians?". Here, the external knowledge required to generate the output will be present in the provided KB. A common way to store such facts is in triple format. For example, in this case, the KB may contain triples such as (great Italians, type, restaurant), (great Italians, food, Italian), and (great Italians, price, cheap), etc. Given the above two utterances, the user may say "That sounds good, can you book a table for 4 at 7pm?", for which the assistant will perform the system action book_table(name = great Italians, num_seats = 4, time = 7pm) and generate a text response "Done!".

[0025] The implementation disclosed herein enables direct mapping of conversation / dialogue history and KB to next-turn system actions and textual responses without any intermediate symbolic states and without intermediate supervisory signals.

[0026] The dialogue / conversation history used in this paper includes user utterances and / or system utterances, and is updated after each round of user-automated assistant dialogue to reflect the contribution of the most recent round to the dialogue history. More formally, let ((u 1 , a 1 ), (u 2 , a 2 ),…,(u U , a U )) indicates that for user utterance (u i ) and assistant speech (a i ) contains a conversation history of U turns. User and assistant turns each contain a variable number of word tokens.

[0027] The knowledge base / knowledge source used herein includes a collection of discrete structured and / or unstructured resources. For example, the knowledge base may include a knowledge graph, which includes triples (or other representations), each triple defining a pair of entities and the relationship between the pair of entities. Moreover, for example, the knowledge base may additionally or alternatively include all or part of an unstructured web-based document (e.g., a title, a fragment, a first sentence), unstructured search results, and / or other resources. In various implementations, the knowledge base considered at runtime may be static in multiple (e.g., all) dialogue sessions and / or dialogue turns. In other implementations, the knowledge base considered in a given turn may be dynamically determined based on all or part of the conversation history to that point (e.g., at least the most recent user utterance). For example, in a given turn, a search may be issued to identify one or more discrete resources in response to the search, and the knowledge base of a given turn may be based on one or more (e.g., the first 10 or other number) of the identified discrete resources. For example, a discrete resource may be all or part of a corresponding response search result, a part of a corresponding response search result document, or a corresponding response knowledge graph entry (e.g., a triple), respectively.

[0028] For the implementation of a structured KB containing a list of triples, it can be formally expressed as Where e stands for entity and r stands for relationship. Entities and / or relationships can be represented by or associated with corresponding text (i.e., aliases of entities).

[0029] In the case where the neural network model includes a decoder (e.g., a transformer decoder), it can be used to generate both the action and the text response from left to right one token at a time. The system action can also be in text format, and a concatenated version of the system action and text response can be generated as one long sequence. More formally, let Y = (Y1, Y2, ..., y T ) represents the target sequence. The target sequence distribution can be modeled as:

[0030]

[0031] Some implementations disclosed herein can utilize a transformer sequence-to-sequence model that is used to process the concatenation of conversational history and KB. The transformer decoder is used to generate system actions and text responses for the assistant's next turn. Given a training set of N examples ((x 1 , K 1 ,y 1 ),(x 2 ,K 2 ,y 2 ),...,(x N ,K N ,y N)), the objective function to be maximized is given by

[0032]

[0033] Although the transformer sequence-to-sequence model with the aforementioned objective function can be effectively utilized in various implementations, other implementations disclosed herein utilize alternative approaches. The alternative approaches can reduce the time and / or amount of computational resources during training, can alleviate overfitting problems during training, and / or alleviate the inclusion of relatively rare tokens in the knowledge base.

[0034] Now we describe some alternative approaches. Some implementations use a transformer encoder to process the conversation history instead of using a transformer encoder to process the knowledge base. More formally, let x = (x1, x2, ..., x p ) is a concatenated conversation history containing P tokens (e.g., assistant and user turns separated by a delimiter). By processing the conversation history with the transformer encoder, P hidden states h1h2, ..., h can be generated after word embedding lookup and multiple self-attention layers of the transformer encoder. p Each discrete resource in the knowledge base can be represented based on the generated word embeddings of the tokens for the discrete resource. For example, a KB triple can be represented as the average of the word embeddings of the tokenized triples. More formally, for a KB triple They can be represented as v1, v2, ..., v M

[0035] The transformer decoder can continue to be used and can include self-attention layers and encoder-decoder attention layers. In various implementations, overfitting and / or other issues are mitigated by adopting a sparse attention mechanism on the KB. For example, the average of the encoder hidden state Can be used to calculate the S-shaped probability for each fact In addition, if q i If is less than a threshold, it can be set to zero, which limits the number of electronic resources that the model can use to fit a given training example.,Sigmoid attention can optionally be used instead of the regular softmax attention, as it,makes it easier to select more than one electronic resource.

[0036] The transformer decoder can be modified to process both the encoder hidden states of the conversation history and the representation of fact triples. In other words, the decoder attention head processes the set [h1, ..., h P , v1, ..., v M ], rather than just processing [h1, ..., h P]. When applying decoder attention, attention fusion can be performed by modifying the initial representation of discrete resources based on the sparse attention probability. For example, the logarithm of a fact triple can be modified by multiplying the logarithm by the sparse sigmoid attention probability.

[0037] As described above, distant supervision can optionally be utilized, where a discrete resource can be labeled as positive at training time if one or more terms in the discrete resource match one or more terms in the ground truth response and / or action. For example, if certain words in the entity of the triple (e 1 , e 2 in(e 1 ,r,e 2 )) is present in the ground truth responses and / or actions, the triplet can be labeled as positive. This weak supervisory signal may guide the sparse attention mechanism described above. Additionally, to increase training speed, negative sampling can be utilized, where among all (weakly) labeled negative samples (i.e., those that are not labeled as positive), a small number of them can be sampled and added as input. At runtime, since the ground truth responses and / or actions are not available, the model can optionally be used to process the entire KB as input.

[0038] The distant supervision objective to be maximized can be formally expressed as:

[0039]

[0040] where q m is the sparse S-shaped attention probability, y m is an indicator variable which is set to 1 if some words in the triple entity exist in the ground truth target sequence and set to 0 otherwise.

[0041] The model can be trained by maximizing the interpolation of the two objective functions in formula (2) and formula (3), which is given by:

[0042]

[0043] Processing with the transformer decoder can generate a corresponding probability distribution over the entire vocabulary at each time step. In task-oriented dialogues, many tokens in text responses, such as entity names, are obtained from the KB. Since these entities appear rarely, it may be difficult for the model to assign high probabilities to such rare words. With these considerations in mind, the implementation can enhance the decoder by allowing it to copy from the input or generate from the vocabulary at each time step. The probability of generating a word from the vocabulary is calculated as described above. The probability of copying is calculated by averaging the multi-head decoder attention distribution (conversation history + KB) over the input. The final probability for generating a word is obtained as a weighted mixture of generating and copying the word. To obtain this weight, at each step in the decoding, the model calculates a sigmoid probability for generating a token from the vocabulary at that time step.

[0044] Output sequences containing actions and text responses are often long. In this case, teacher forcing training may lead to poor performance. In teacher forcing, training is conditioned on the ground truth history, which is replaced by the model predictions at test time. If the model makes a mistake when decoding the beginning of the output sequence, it may cause the model to be conditioned on history that it has never seen during training. Scheduled sampling aims to mitigate the difference in training and test time by making the model condition on its own predictions during training.

[0045] In various implementations, in order to overcome the difficulty of training a sparse attention mechanism, the sparse attention mechanism may not be trained for the first M (e.g., 20,000 or other number) steps of model training. For example, during those steps, the converter encoder and / or converter decoder weights may be updated without updating the weights of the sparse attention mechanism. For the first M steps, positively labeled electronic resources from remote supervision can be used as electronic resources, and the decoder can process those electronic resources. These initial M training steps will warm-start the model and train other parts of the model except for attention. For the next N steps (e.g., 20,000 or other number), the sparse attention mechanism will be trained. In addition, the sparse attention to the KB will be inserted between the remote supervision and the sparse S-shaped probability generated from the model using the sparse attention mechanism. For example, when determining the electronic resource-based attention to the decoder, for a positively labeled electronic resource, it can be based on the insertion between the sparse S-shaped probability of the resource and the positive (e.g., "1") probability of the electronic resource inferred from the remote supervision. In some implementations, the weight on the distant supervision is gradually decayed (e.g., from 1.0 to 0.0) over N steps. This can gradually increase the training impact of errors attributable to the sparse attention mechanism. For the last O (e.g., 10,000 or other number) steps, the sparse attention mechanism is continued to be trained. However, only the sparse attention mechanism is used, without considering distant supervision (or with little distant supervision).

[0046] Turning now to the drawings, these and other implementations are described in greater detail.

[0047] Figure 1 An example environment in which the techniques disclosed herein may be implemented is shown. The example environment includes a client device 106, an automated assistant 110, and a plurality of agents 140A-N. The client device 106 may be, for example, a standalone voice-controlled speaker device, a desktop computing device, a laptop computing device, a tablet computing device, a mobile phone computing device, a computing device of a user's vehicle, and / or a wearable device of a user including a computing device (e.g., a watch of the user with the computing device, glasses of the user with the computing device, a virtual reality or augmented reality computing device). Additional and / or alternative client devices may be provided.

[0048] Despite Figure 1 106, in some implementations, all or aspects of automated assistant 110 may be implemented by client device 106. For example, in some implementations, input processing engine 112 and / or response and action generation engine 124 may be implemented by client device 106. In some implementations, where one or more (e.g., all) aspects of automated assistant 110 are implemented by one or more computing devices remote from client device 106, client device 106 and those aspects of automated assistant 110 communicate via one or more networks, such as a wide area network (WAN) (e.g., the Internet).

[0049] Although only one client device 106 is shown in conjunction with the automated assistant 110, in many implementations, the automated assistant 110 can be remote and can interface with each of multiple client devices of the same user and / or with each of multiple client devices of multiple users. For example, the automated assistant 110 can manage communications with each of the multiple devices via different sessions and can manage multiple sessions in parallel. For example, in some implementations, the automated assistant 110 can be implemented as a cloud-based service that employs a cloud infrastructure (e.g., using a server farm or high-performance computer cluster running software suitable for handling a large number of requests from multiple users). However, for simplicity, many examples herein are described with respect to a single client device 106.

[0050] The automated assistant 110 is separate from the agents 140A-N and communicates with the agents 140A-N via an API and / or via one or more communication channels (e.g., an internal communication channel of the client device 106 and / or a network, such as a WAN). In some implementations, each of the one or more agents 140A-N is managed by a respective party that is separate from the party that manages the automated assistant 110. As used herein, an "agent" refers to one or more computing devices, smart devices, and / or software that can be used by the automated assistant to cause the execution of actions generated by the response and action generation engine 124. In some cases, the agent can be separate from the automated assistant 110 and / or can communicate with the automated assistant 110 via one or more communication channels. In some of these cases, the automated assistant can transmit data (e.g., agent commands) from a first network node to a second network node that implements all or various aspects of the agent's functionality. In some cases, the agent can be a third party (3P) agent because it is managed by a party that is separate from the party that manages the automated assistant 110. In some other cases, the agent can be a first party (1P) agent because it is managed by the same party that manages the automated assistant 110.

[0051] The agent is configured to receive an action command sent by the automated assistant 110 (e.g., over a network and / or via an API). In response to receiving the action command, the agent performs the corresponding action and optionally generates response content for providing a user interface output based on the response content. For example, the agent can transmit the response content to the automated assistant 110 so that the automated assistant 110 provides an output based on the response content. As another example, the agent itself can provide output. For example, a user can interact with the automated assistant 110 via a client device 106 (e.g., the automated assistant can be implemented on the client device 106 and / or communicate with the client device 106 over a network), and the agent can be an application installed on the client device 106 or an application executable away from the client device 106, but "streamable" on the client device 106. When the application is called, it can be executed by the client device 106 and / or placed in the forefront by the client device 106 (e.g., its content can take over the display of the client device 106).

[0052] As an example, one of the agents 140A-N may be a smart device, or a third-party system that controls the smart device, and the action command may be a command that causes the state of the smart device to be changed. In response to receiving the action command, the agent may cause the state of the smart device to change, and may optionally provide response content that confirms the change of state. As another example, one of the agents 140A-N may be a ride-sharing reservation system, and the action command may be a command for causing ride-sharing to be booked. In response to receiving the action command, the agent may cause ride-sharing to be booked, and may optionally provide response content that confirms the reservation and / or provides an estimated vehicle arrival time. As yet another example, one of the agents may be a separate application on the client device 106, and the action command may be a command that causes a separate application to be opened and / or controlled in a particular manner. In response to receiving the action command, a separate application may be opened and / or controlled according to the action command.

[0053] The automated assistant 110 includes an input processing engine 112, an output engine 116, an action engine 118, and a response action generation engine 124. Figure 1 In some implementations, automated assistant 110 does not include a natural language understanding component and does not include a fulfillment module because those modules are omitted in view of response and action generation engine 124. In some implementations, some engines (and / or engines) of automated assistant 110 may be omitted, combined, and / or implemented in components separate from automated assistant 110. Furthermore, for simplicity, automated assistant 110 may include other engines (and / or modules) not shown here.

[0054] Automated assistant 110 receives an example of user input from client device 106. The user can provide various types of user input to automated assistant 110 via user interface input devices. In some cases, the input can be a "user utterance" of free-form natural language input, such as text input based on user interface input generated by the user via one or more user interface input devices (e.g., based on typed input provided via a physical or virtual keyboard or based on spoken input provided through a microphone). As used herein, free-form input is formulated by the user and is not limited to presenting a set of options for the user to select (e.g., not limited to a set of options presented in a drop-down menu).

[0055] As an example, the automated assistant 110 may receive spoken user utterances in the form of a streaming audio recording. The streaming audio recording may be generated by the client device 106 in response to a signal received from a microphone of the client device 106 that captures the spoken input of the user of the client device 106. In some implementations, the user input may be generated by the client device 106 and / or provided to the automated assistant 110 in response to an explicit invocation of the automated assistant 110 by the user of the client device 106. For example, the invocation may be a detection by the client device 106 of certain voice inputs from the user (e.g., a hot word / phrase of the automated assistant 110, such as "Hey assistant"), user interaction with a hardware button and / or a virtual button (e.g., clicking a hardware button, selection of a graphical interface element displayed by the client device 106), and / or other specific user interface inputs. When the user input is in the form of spoken utterances, the automatic speech recognition (ASR) module 113 of the input processing engine 112 may convert the spoken utterances into recognized text using a speech recognition model. The input processing engine 112 may optionally normalize or otherwise convert some of the text. For example, the input processing engine 112 can remove uppercase letters, convert dates to a standard format, convert relative time items to a standard format (e.g., converting "tomorrow" to the date corresponding to "tomorrow"), convert numbers to universal identifiers (e.g., converting any number to "#"), etc.

[0056] The response and action generation engine 124 uses the neural network model 160 to directly generate natural language responses and / or actions in response to the received user utterances. The response and action generation engine 124 generates responses and / or actions based on processing the following items using the neural network model: (a) discrete resources from the knowledge base 158 and / or the search engine 154, and (b) the received user utterances, and any previous user and / or automated assistant utterances of the current dialogue session. For example, the response and action generation engine 124 can generate a dialogue encoding based on processing the received user utterances and any previous utterances of the dialogue session using an encoder of the neural network model 160. In addition, for example, the response and action generation engine 124 can generate a resource encoding based on processing a representation of the resource using an encoder (together with the dialogue history) or based on a representation of the resource generated using a probabilistic modification of the sparse attention mechanism using the neural network model 160. Additional description of the response and action generation engine 124 and the neural network model 160 is provided here.

[0057] As used herein, a "conversation session" may include a logically separate exchange of one or more messages between a user and automated assistant 110 (and, in some cases, other human participants) and / or the performance of one or more responsive actions by automated assistant 110. Automated assistant 110 may distinguish between multiple conversation sessions with a user based on various signals, such as the passage of time between sessions, changes in user context (e.g., location, before / during / after a scheduled meeting) between sessions, detection of one or more intermediate interactions between the user and a client device other than a conversation between the user and the automated assistant (e.g., the user switches apps for a period of time, the user leaves and then returns to a standalone voice-activated product), locking / sleeping of client devices between sessions, and changes in client devices used to interface with one or more instances of automated assistant 110, etc.

[0058] The output engine 116 generates output based on the natural language responses generated by the response and action generation engine 124. For example, the output engine 116 can provide the generated natural language responses for graphical presentation at the client device 106, and / or the text-to-speech (TTS) module 117 can perform TTS conversion of the responses and provide corresponding synthesized audio data at the client device 106 for audible rendering of the natural language responses.

[0059] The action engine 118 causes execution of any action generated by the response and action generation engine 124. For example, the action engine 118 may identify a given agent in the agents 140A-N based on its correspondence to the intent and / or other parameters specified by the action. In addition, the action engine 118 may transmit a corresponding action command to the given agent. In some implementations, the action command may include (or be limited to) the action generated by the response and action generation engine 124. In some alternative implementations, the action generated by the response and action generation engine 124 may be optionally changed (e.g., with respect to format or form), and the changed action is included in the action command.

[0060] Figure 2A Shows Figure 1 Implementation of the neural network model 160A and Figure 1 The response and action generation engine 124 can utilize it to generate natural language responses and optionally actions 177A. The response and action generation engine 124 can generate the dialogue encoding and resource encoding based on processing the dialogue history 171A and electronic resources from the knowledge base 158 and / or the search engine 154 using the transformer encoder 160A1 of the neural network model 160A. In addition, the response and action generation engine 124 can generate the NL response and optionally the action 177A token by token using the transformer decoder 160A2 that processes the dialogue encoding and resource encoding.

[0061] Figure 2B Shows Figure 1 Another implementation of the neural network model 160B and Figure 1 How the response and action generation engine 124 utilizes it to generate a natural language response and optional action 177B. Figure 2B The implementation 160B of the neural network model includes: a transformer encoder 160B1, a sparse attention mechanism 160B2, and a transformer decoder 160B3. The response and action generation engine 124 can generate a dialogue encoding 172B based on processing the dialogue history 171B using the transformer encoder 160B1. In addition, the response and action generation engine 124 can generate probabilities 173B of discrete resources of the knowledge base 158 based on processing the dialogue encoding 172B using the sparse attention mechanism 160B2 and optionally processing discrete resources (e.g., their representations). Further, the response and action generation engine 124 can generate natural language responses and optionally actions 177B token by token using a transformer decoder 160B3 that processes both the dialogue encoding 172B and the attention fusion 174B. The attention fusion 174B can be a fusion of the initial resource representations of the discrete resources with their corresponding probabilities 173B (thereby representing the encoded representations of the discrete resources).

[0062] Figure 3 is a flow chart illustrating an example method 300 for generating natural language responses and optionally actions directly from a conversation history and discrete resources. For convenience, reference is made to the system description for performing the operations Figure 3 The system may include various components of various computer systems, such as the response and action generation engine 124 of the automated assistant 110. In addition, although the operations of the method 300 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added.

[0063] At block 350 , the system receives a spoken user utterance. For example, the spoken user utterance may be received in the form of audio data generated by one or more microphones of a user's client device.

[0064] At block 352, the system performs ASR to convert the spoken utterance into recognized text. At block 352, the system also adds the recognized text to the conversation history of the current conversation session. Although blocks 350 and 352 are shown in method 300, it should be understood that in various implementations, the received user input may be text input (e.g., via typing), thereby eliminating the need to perform ASR at block 352.

[0065] At block 354, the system issues a search based on the conversation history of the current conversation session. For example, the system can issue a search based on one or more terms identified in a recently received user utterance and / or based on one or more terms in previous user and / or assistant utterances (if any) in the history of the current conversation session. For example, the system can issue a search based on all terms of the user utterance or certain terms (e.g., terms that are not classified as stop words). A search can be issued by submitting a term to the search system.

[0066] At block 356, the system identifies one or more discrete resources based on the discrete resources in response to the search issued at block 354. For example, each discrete resource can be based on all or a portion of a corresponding search result, all or a portion of a corresponding search result document, and / or a corresponding knowledge graph entry. For example, the system can identify a portion of the titles and / or snippets of all or a portion of the top ten (or other number) search results as discrete resources (i.e., discrete resource 1 based on search result 1, discrete resource 2 based on search result 2, etc.).

[0067] At block 358, the system processes the concatenation of the dialogue history and discrete resources using a transformer encoder of a neural network to generate an encoded dialogue representation and an encoded resource representation. The encoded dialogue and resource representations can be a series of hidden states after word embeddings and multiple self-attention layers.

[0068] At block 360, the system generates a probability distribution over the token vocabulary of the neural network using a transformer decoder that processes both the encoded dialogue representation and the encoded resource representation (via multi-head attention). As described herein, in some implementations, for tokens in the dialogue history and / or discrete resources, the system can effectively improve the probability of selecting a token. For example, the system can determine its probability as a weighted mixture of the generated probability distribution over the input and the multi-head attention distribution.

[0069] At block 362 , the system selects a token from the vocabulary based on the probability distribution.

[0070] At block 364, the system determines whether response and action generation has ended, based on, for example, whether the most recent token of block 362 is an "end" token (if so, the decision of block 364 is "yes"). If not, the system adds the most recently selected token as input to the decoder (along with any tokens selected in the previous iteration of block 362) and returns to block 360. This process continues until an end condition is determined at block 364, thereby generating natural language responses and / or actions on a token-by-token basis.

[0071] When an end condition is determined at block 364, the system proceeds to block 368 and causes the token's response to be rendered (eg, graphically and / or audibly as synthesized speech) and / or the token's action to be performed.

[0072] At box 370, the response of box 368 and optionally the action (if any) are added to the conversation history. The system then returns to box 350 after receiving another user utterance. Note that some of the boxes of 368 may result in the end of the conversation session (e.g., if the action is an end action), in which case box 370 may be skipped, the conversation session is considered to be ended and the conversation session is cleared. The method 300 may then be executed in response to receiving another user utterance in a new conversation session. In addition, even if the end of the conversation session is not determined at box 368, the failure to receive another user utterance within a threshold amount of time may also result in the conversation session being considered terminated.

[0073] Figure 4 is another flow chart illustrating an example method 400 for generating natural language responses and optionally actions directly from a conversation history and discrete resources. For convenience, reference is made to the system description for performing the operations Figure 4 The system may include various components of various computer systems, such as the response and action generation engine 124 of the automated assistant 110. In addition, although the operations of the method 400 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added.

[0074] At block 450, the system receives a spoken user utterance. For example, the spoken user utterance may be received in the form of audio data generated by one or more microphones of a user's client device.

[0075] At box 452, the system performs ASR to convert the spoken utterance into the recognized text. At box 452, the system also adds the recognized text to the conversation history of the current conversation session. Although boxes 450 and 4352 are shown in method 400, it should be understood that in various implementations, the received user input can be text input (e.g., by typing), thereby eliminating the need to perform ASR at box 452.

[0076] At block 454, the system processes the conversation history using the transformer encoder to generate an encoded conversation representation. For example, the encoded conversation representation can be a series of hidden states after word embeddings and multiple self-attention layers. The conversation history can be processed using the transformer encoder without processing any discrete resources using the transformer encoder.

[0077] At block 456, the system performs sparse attention on the discrete resources based on the encoded dialogue representation. For example, the system can use a trained sparse attention mechanism and perform sparse attention based on an average or other metric of a series of hidden states of the dialogue representation. Each of the discrete resources can be, for example, a knowledge graph entry. When performing sparse attention, an initial representation of the discrete resources can optionally be considered. At block 456, the system can generate a corresponding probability (e.g., an S-shaped probability) for each discrete resource.

[0078] At block 458, the system generates an encoded resource representation based on the probabilities generated from the sparse attention of block 456. For example, an initial representation of the discrete resource can be modified based on the probabilities of the discrete resource and the modified representation can be used as the encoded resource representation. The initial representation of the discrete resource can be, for example, an average of word embeddings of the discrete resource or other representations.

[0079] At block 460, the system generates a probability distribution over the token vocabulary of the neural network using a transformer decoder that processes both the encoded dialogue representation and the encoded resource representation (via multi-head attention). As described herein, in some implementations, for tokens in the dialogue history and / or discrete resources, the system can effectively improve the probability of selecting a token.

[0080] At block 462 , the system selects a token from the vocabulary based on the probability distribution.

[0081] At block 464, the system determines whether response and action generation has ended, which can be based on, for example, whether the latest token of block 462 is an "end" token (if so, the determination of block 464 is "yes"). If not, the system adds the most recently selected token as input to the decoder (as well as any tokens selected in the previous iteration of block 462) and returns to block 460. This process continues until an end condition is determined at block 464, thereby generating natural language responses and / or actions on a token-by-token basis.

[0082] When an end condition is determined at block 464, the system proceeds to block 468 and causes the token's response to be rendered (eg, graphically and / or audibly as synthesized speech) and / or the token's action to be performed.

[0083] At block 470, the response of block 468 and optionally the action (if any) are added to the conversation history. The system then returns to block 450 after receiving another user utterance. Note that some of the blocks of 468 may result in the end of the conversation session (e.g., if the action is an end action), in which case block 470 may be skipped, the conversation session may be considered to be ended, and the conversation session may be cleared. Method 400 may then be executed in response to receiving another user utterance in a new conversation session. In addition, even if the end of the conversation session is not determined at block 468, the failure to receive another user utterance within a threshold amount of time may also result in the conversation session being considered terminated.

[0084] Figure 5 is a flowchart illustrating an exemplary method 500 for training a single neural network model that can be used to generate natural language responses and optionally actions directly from a conversation history and discrete resources. For convenience, reference is made to the system description of performing operations. Figure 5 The system may include various components of various computer systems, such as multiple GPUs. In addition, although the operations of method 500 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added.

[0085] At box 552, the system obtains a training instance with a ground truth current dialogue history and a ground truth next action.

[0086] At block 554, the system selects a subset of positive discrete resources for the training instance. The system selects the subset based on the content of the selected discrete resources matching the ground truth next response and / or ground truth next action of the training instance.

[0087] At optional block 556, the system selects a subset of negative discrete resources for the training instance. For example, the system can randomly sample the number of remaining resources from the discrete resources not selected at block 554 and select that number as the weakly labeled negative discrete resources. In some implementations, block 556 is not performed for the first N iterations of method 500, but is performed for all iterations of the first N iterations. In some implementations (e.g., when training a model used in method 300), block 556 is omitted.

[0088] At block 558, the system generates an encoded dialog representation using the encoder, and generates an encoded resource representation. The system generates the encoded dialog representation by processing the ground truth current dialog history using the encoder. In some implementations (e.g., when training a model for use in method 300), the system also generates an encoded resource representation by processing the selected positive discrete resources and the ground truth current dialog history using the encoder.

[0089] At block 560, the system generates a predicted next natural language response and / or predicted action token by token and using the decoder of the encoded dialog and resource representations of processing blocks 556 and 558. This may be similar to blocks 360, 362, 364, and 366 of method 300, and may optionally utilize scheduled sampling as described herein.

[0090] At box 562, the system updates at least the weights of the encoder and decoder based on a comparison of the predicted natural language responses and / or actions with the ground truth natural language responses and / or actions of the training instances.

[0091] In optional box 564, the system updates the sparse attention mechanism (if any sparse attention mechanism is included in the model). In some implementations and / or for certain iterations, the weights of any sparse attention mechanism of the model can also be updated based on the comparison at box 562. In some of those implementations, the weights of the sparse attention mechanism are not updated for the first N iterations of method 500. In some implementations, the weights of the sparse attention mechanism are additionally or alternatively generated based on a comparison of weak markings of positron resources and negative electron resources with corresponding probabilities generated using the sparse attention mechanism. In some of those implementations, the weights of the encoder and / or decoder can also be generated based on such comparison.

[0092] At block 566, the system determines whether to process additional training instances. If so, the system proceeds to block 552, obtains additional training instances, and then performs blocks 554, 556 and 558, 560, 562, and 564 based on the additional unprocessed training instances. In some implementations, at block 552, if one or more training criteria have been met (e.g., a threshold number of epochs have occurred and / or a threshold duration of training has occurred), the system may determine not to process any additional unprocessed training instances.

[0093] If at block 566 the system determines not to process further training instances, the system proceeds to block 568 and training ends. The trained model may then be used in method 300 or method 400 (depending on the construction of the model and the training performed).

[0094] Figure 6 610 , which may optionally be used to perform one or more aspects of the techniques described herein. In some implementations, one or more of the client device 106 , the automated assistant 110 , and / or other components may include one or more components of the example computing device 610 .

[0095] The computing device 610 typically includes at least one processor 614 that communicates with a number of peripheral devices via a bus subsystem 612. These peripheral devices may include a storage subsystem 624 including, for example, a memory subsystem 625 and a file storage subsystem 626, a user interface output device 620, a user interface input device 622, and a network interface subsystem 616. The input and output devices allow a user to interact with the computing device 610. The network interface subsystem 616 provides an interface to an external network and couples to corresponding interface devices in other computing devices.

[0096] The user interface input device 622 may include a keyboard, a pointing device such as a mouse, a trackball, a touch pad or a graphics tablet, a scanner, a touch screen incorporated into a display, an audio input device such as a spoken word recognition system, a microphone, and / or other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and ways to input information into the computing device 610 or a communication network.

[0097] User interface output devices 620 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or some other mechanism for creating a visible image. The display subsystem may also provide a non-visual display, for example, via an audio output device. In general, the use of the term "output device" is intended to include all possible types of devices and ways of outputting information from computing device 610 to a user or another machine or computing device.

[0098] The storage subsystem 624 stores programs and data structures that provide the functionality of some or all of the modules described herein. For example, the storage subsystem 624 may include logic to perform selected aspects of the methods described herein.

[0099] These software modules are typically executed by the processor 614 alone or in conjunction with other processors. The memory 625 used in the storage subsystem 624 may include multiple memories, including a main random access memory (RAM) 630 for storing instructions and data during program execution and a read-only memory (ROM) 632 in which fixed instructions are stored. The file storage subsystem 626 may provide persistent storage for program and data files, and may include a hard drive, a floppy disk drive and associated removable media, a CD-ROM drive, an optical drive, or a removable media cartridge. Modules that implement certain implemented functions may be stored in the storage subsystem 624 by the file storage subsystem 626, or in other machines accessible to the processor 614.

[0100] The bus subsystem 612 provides a mechanism by which the various components and subsystems of the computing device 610 communicate with each other as intended. Although the bus subsystem 612 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple busses.

[0101] The computing device 610 can be of various types, including a workstation, a server, a computing cluster, a blade server, a server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, for the purpose of illustrating some implementations, the present invention is described in detail below. Figure 6 The description of computing device 610 depicted in FIG. 6 is intended only as a specific example. Many other configurations of computing device 610 may have more Figure 6 The computing devices depicted may have greater or fewer components.

[0102] In cases where the systems described herein collect or otherwise monitor personal information about a user, or where personal and / or monitored information may be used, the user may be provided with the opportunity to control whether a program or feature collects information about the user (e.g., information about the user's social network, social activities or behavior, occupation, the user's preferences, or the user's current geographic location) or to control whether and / or how content that may be relevant to the user is received from a content server that may be relevant to the user. In addition, certain data may be processed in one or more ways prior to storage or use so as to remove personally identifiable information. For example, the identity of a user may be processed so that no personally identifiable information can be determined for that user, or where geographic location information is obtained, the user's geographic location may be generalized (e.g., to a city, zip code, or state level) so that the user's specific geographic location cannot be determined. Thus, the user may have control over how information about the user is collected and / or used.

[0103] In some implementations, a method is provided that includes generating recognized text from spoken utterances of a user using an automatic speech recognition component during a conversation session. The conversation session is between a user and an automated assistant, and the spoken utterances are captured in audio data detected by one or more microphones of a client device. The method also includes adding the spoken utterances to a conversation history of the conversation session. The method also includes using a converter encoder to generate an encoded conversation representation based on processing the recognized text and any other text of the conversation history of the conversation session. The method also includes: identifying a plurality of discrete resources; and generating an encoded resource representation based on processing a plurality of discrete resources. The method also includes generating a natural language response and / or action in response to the spoken utterance. Generating the natural language response and / or action includes generating the natural language response and / or action token by token using a converter decoder that processes both the encoded conversation representation and the encoded resource representation. The method also includes, as a response of the automated assistant to the spoken utterance, rendering the natural language response at the client device and / or causing the action to be performed.

[0104] These and other implementations disclosed herein may include one or more of the following features.

[0105] In some implementations, generating an encoded resource representation includes: determining a corresponding probability for each of a plurality of discrete resources based on the encoded conversation representation; and, generating the encoded resource representation based on the corresponding initial representations of the plurality of discrete resources and the corresponding probabilities. In some versions of those implementations, determining the corresponding probability for each of the plurality of discrete resources based on the encoded conversation representation includes using a sparse attention mechanism when determining the corresponding probabilities. In some versions of those versions, using the sparse attention mechanism to determine the corresponding probability for each of the plurality of discrete resources includes: processing an average of the encoded conversation representation using the sparse attention mechanism. Generating the encoded resource representation based on the initial representations of the plurality of discrete resources and the corresponding probabilities may optionally include: multiplying the initial representations by their corresponding probabilities. Each of the corresponding initial representations may, for example, be based on a corresponding word embedding of the corresponding discrete resource in the discrete resources.

[0106] In some implementations, each of the discrete resources is a corresponding triple from the knowledge graph.

[0107] In some implementations, the conversation history of the conversation session also includes one or more previous automated assistant responses in the conversation session and / or one or more previous instances of recognized text from previous user utterances in the conversation session.

[0108] In some implementations, generating the encoded resource representation based on processing the plurality of discrete resources includes processing the plurality of discrete resources using a transformer encoder along with the recognized text and any other text of the conversation history.

[0109] In some implementations, identifying the plurality of discrete resources includes: issuing a search based on the identified text; and identifying the plurality of discrete resources based on the plurality of discrete resources being responsive to the issued search.

[0110] In some implementations, generating a natural language response and / or action token by token using a transformer decoder that processes both the encoded conversation representation and the encoded resource representation includes: generating a probability distribution over a vocabulary of tokens at each of a plurality of iterations of processing using the transformer decoder; selecting a corresponding token from the vocabulary of tokens for the iteration based on the probability distribution over the iteration; and using the selected token as part of the natural language response and / or action and as part of the input to the transformer decoder at a subsequent iteration of the processing. Some of these implementations also include: changing the probability distribution by changing the token probabilities of one or more tokens in the vocabulary that are also included in the conversation history and / or the discrete resource.

[0111] In some implementations, using a transformer decoder that processes both the encoded dialog representation and the encoded resource representation, generating the natural language response and / or action token by token includes: generating a sequence of tokens that includes the natural language response and the action. Some versions of these implementations include transmitting the action to a remote device to cause the action to be performed. The action can, for example, cause a state of a smart device to be changed, and wherein the remote device is a smart device, or a remote server that controls the smart device.

[0112] In some implementations, a method is provided, the method comprising obtaining, for a training instance: a ground truth current conversation history of a conversation session with an automated assistant, and a ground truth next response and / or a ground truth next action of the conversation session. The method further comprises selecting a subset of positive discrete resources for the training instance from a set of candidate discrete resources. Selecting the subset of positive discrete resources may be based on each of the discrete resources including corresponding content that matches the ground truth next response and / or the ground truth next action. The method further comprises generating an encoded conversation representation based on processing the conversation history using a transformer encoder. The method further comprises generating a predicted next natural language response and / or a predicted next action token by token based on the subset of positive discrete resources using a transformer decoder that processes both the encoded conversation representation and the encoded resource representation. The method further comprises updating weights of at least a transformer encoder and a transformer decoder based on a comparison of (a) the predicted next natural language response and / or the predicted next action with (b) the ground truth next response and / or the ground truth next action of the conversation session.

[0113] These and other implementations disclosed herein may include one or more of the following features.

[0114] In some implementations, the method further includes selecting a subset of negative discrete resources for the training instance from the set of candidate discrete resources that are not in the subset. In those implementations, the encoded resource representation can also be based on processing the subset of negative discrete resources. In some of those implementations, selecting the subset of negative discrete resources for the training instance is responsive to determining that a threshold number of training steps have occurred and / or a threshold duration of training has occurred.

[0115] In some implementations, the method also includes determining to inhibit updating any weights of the sparse attention mechanism in response to determining that a threshold number of training steps have not occurred and / or a threshold duration of training has not occurred.

[0116] In some implementations, the method also includes generating an encoded resource representation. Generating the encoded resource representation may include: determining a corresponding prediction probability for each positive discrete resource of the subset based on processing the encoded dialogue representation using a sparse attention mechanism; determining a modified corresponding prediction probability for each positive discrete resource of the subset based on the corresponding prediction probability and the positive probability; and, generating the encoded resource representation based on the corresponding initial representation of the positive discrete resources of the subset and the modified corresponding prediction probability. The positive probability can be used to generate the modified corresponding prediction probability based on each of the positive discrete resources including corresponding content that is also included in the ground truth next response and / or the ground truth next action. In some versions of those implementations, the weighting of the positive probability in determining the modified corresponding prediction probability is based on the number of training steps that have occurred and / or the duration of training that has occurred.

[0117] In some implementations, the method further includes generating an encoded resource representation. Generating the encoded resource representation may include: determining a corresponding predicted probability for each positive discrete resource of the subset based on processing the encoded dialogue representation using the sparse attention mechanism; generating the encoded resource representation according to the corresponding initial representations and the unmodified corresponding predicted probabilities of the positive discrete resources of the subset. In some versions of those implementations, generating the encoded resource representation according to the corresponding initial representations and the unmodified corresponding probabilities of the positive discrete resources of the subset is responsive to determining that a threshold number of training steps have occurred and / or a threshold duration of training has occurred. In some implementations, the method further includes updating a weight of the sparse attention mechanism based on a comparison of (a) a predicted next natural language response and / or a predicted next action with (b) a ground truth next response and / or a ground truth next action of the dialogue session. Updating the weight of the sparse attention mechanism may optionally also be based on a comparison of the corresponding predicted probabilities with the positive probabilities. Updating the weight of the converter encoder and / or the converter decoder may optionally also be based on a comparison of the corresponding predicted probabilities with the positive probabilities.

[0118] In some implementations, each of the discrete resources is a corresponding triple from the knowledge graph.

Claims

1. A method implemented by one or more processors, the method comprising: generating recognized text from spoken utterances of the user captured in audio data detected by one or more microphones of the client device using an automatic speech recognition component during a conversational session between the user and the automated assistant; adding the spoken utterance to a conversation history of the conversation session; generating an encoded conversation representation based on processing the recognized text and any other text of the conversation history of the conversation session using a transformer encoder; identifying a plurality of discrete resources, wherein the plurality of discrete resources include knowledge graph entries, search results, and / or search result documents, and wherein identifying the plurality of discrete resources includes: issuing a search based on the identified text; and identifying the plurality of discrete resources based on the plurality of discrete resources in response to the issued search; generating an encoded resource representation based on processing the plurality of discrete resources; and Generating a natural language response and / or an action in response to the spoken utterance, generating the natural language response and / or the action comprises: generating said natural language response and / or said action token by token using a transformer decoder processing both said encoded dialog representation and said encoded resource representation; and The natural language response is caused to be rendered and / or the action is performed at the client device as a response of the automated assistant to the spoken utterance.

2. The method according to claim 1, wherein: Generating the encoded resource representation includes: determining a corresponding probability for each of the plurality of discrete resources based on the encoded dialog representation; and The encoded resource representation is generated based on the corresponding initial representations of the plurality of discrete resources and the corresponding probabilities.

3. The method according to claim 2, wherein: Determining the corresponding probability for each of the plurality of discrete resources based on the encoded dialog representation includes using a sparse attention mechanism in determining the corresponding probability.

4. The method according to claim 3, wherein: Determining the corresponding probability of each discrete resource in the plurality of discrete resources using the sparse attention mechanism includes: The sparse attention mechanism is used to process the average of the encoded dialogue representation.

5. The method according to claim 2, wherein: Generating the encoded resource representation according to the corresponding initial representations of the plurality of discrete resources and the corresponding probabilities comprises: The initial representations are multiplied by their corresponding probabilities.

6. The method according to claim 2, wherein: Each of the corresponding initial representations is based on a corresponding word embedding of a corresponding discrete resource of the plurality of discrete resources.

7. The method according to claim 1, wherein: The conversation history of the conversation session further includes one or more previous automated assistant responses in the conversation session and / or one or more previous instances of recognized text from previous user utterances in the conversation session.

8. The method according to claim 1, wherein: Generating the encoded resource representation based on processing the plurality of discrete resources includes processing the plurality of discrete resources using the transformer encoder along with the recognized text and any other text of the conversation history.

9. The method according to claim 1, wherein: Generating the natural language response and / or the action token by token using the transformer decoder processing both the encoded dialog representation and the encoded resource representation comprises: generating a probability distribution over a vocabulary of tokens at each of a plurality of iterations of processing using the transformer decoder; selecting a corresponding token from said token vocabulary for use in said iteration based on said probability distribution for said iteration; and The selected token is used as part of the natural language response and / or the action and, at a subsequent iteration of the plurality of iterations of processing, as part of an input to a decoder of the transformer.

10. The method according to claim 9, further comprising: The probability distribution is changed by changing the token probabilities of one or more tokens in the token vocabulary also included in the conversation history and / or the plurality of discrete resources.

11. The method according to claim 1, wherein: Generating the natural language response and / or the action token by token using the transformer decoder processing both the encoded dialog representation and the encoded resource representation comprises: A token sequence is generated that includes the natural language response and the action.

12. The method according to claim 11, further comprising: The action is transmitted to a remote device to cause the action to be performed.

13. The method according to claim 12, wherein: The action causes a state of the smart device to be changed, and wherein the remote device is the smart device, or a remote server that controls the smart device.

14. A method implemented by one or more processors, the method comprising: For the training instance we get: The ground truth current conversation history of the conversation session with the automated assistant, and a ground truth next response and / or a ground truth next action for the dialog session; selecting a subset of positive discrete resources for the training instance from a set of discrete resources, the subset of positive discrete resources being selected based on each of the discrete resources including corresponding content that matches the ground truth next response and / or the ground truth next action, wherein the set of discrete resources includes knowledge graph entries, search results, and / or search result documents; generating an encoded conversation representation based on processing the conversation history using a transformer encoder; generating a predicted next natural language response and / or a predicted next action based on the subset of positive discrete resources on a token-by-token basis using a transformer-decoder processing both the encoded dialog representation and the encoded resource representation; and Updating weights of at least the transformer encoder and the transformer decoder based on a comparison of (a) a predicted next natural language response and / or a predicted next action with (b) a ground truth next response and / or a ground truth next action for the dialog session.

15. The method according to claim 14, further comprising: selecting a subset of negative discrete resources for the training instance from the set of discrete resources not in the subset; Therein, the coded resource representation is also based on processing a subset of the negative discrete resources.

16. The method according to claim 15, wherein: Selecting the subset of negative discrete resources for the training instance is responsive to determining that a threshold number of training steps have occurred and / or a threshold duration of training has occurred.

17. The method according to claim 14, further comprising: In response to determining that a threshold number of training steps have not occurred and / or a threshold duration of training has not occurred, determining to inhibit updating any weights of the sparse attention mechanism.

18. The method of claim 14, further comprising generating the encoded resource representation, wherein: Generating the encoded resource representation includes: determining a corresponding prediction probability of each of the positive discrete resources of the subset based on processing the encoded dialog representation using a sparse attention mechanism; determining a modified corresponding predicted probability for each of the positive discrete resources of the subset according to the corresponding predicted probability and a positive probability, wherein the positive probability is used to generate the modified corresponding predicted probability based on each of the positive discrete resources including corresponding content also included in the ground truth next response and / or the ground truth next action; and The encoded resource representation is generated based on the corresponding initial representations of the positive discrete resources of the subset and the modified corresponding predicted probabilities.

19. The method according to claim 18, wherein: The weighting of the positive probability when determining the modified corresponding predicted probability is based on the number of training steps that have occurred and / or the duration of training that has occurred.

20. The method of claim 14, further comprising generating the encoded resource representation, wherein: Generating the encoded resource representation includes: determining a corresponding prediction probability of each of the positive discrete resources of the subset based on processing the encoded dialog representation using a sparse attention mechanism; The coded resource representation is generated based on corresponding initial representations of the positive discrete resources of the subset and the corresponding predicted probabilities without modification.

21. The method according to claim 20, wherein: Generating the encoded resource representation based on the corresponding initial representations of the positive discrete resources of the subset and the corresponding predicted probabilities without modification is responsive to determining that a threshold number of training steps have occurred and / or a threshold duration of training has occurred.

22. The method of claim 18, further comprising: The weights of the sparse attention mechanism are updated based on a comparison of (a) a predicted next natural language response and / or a predicted next action with (b) a ground truth next response and / or a ground truth next action for the conversation session.

23. The method according to claim 22, wherein: Updating the weights of the sparse attention mechanism is also based on a comparison of the corresponding predicted probability with the positive probability.

24. The method according to claim 23, wherein: Updating the weights of the transformer encoder and the transformer decoder is also based on a comparison of the corresponding prediction probability with the positive probability.

25. A system comprising one or more processors and a memory storing instructions which, when executed by the one or more processors, cause the method of any one of claims 1 to 24 to be performed.

26. A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, cause the method of any one of claims 1 to 24 to be performed.

Images