Reactive learning for efficient dialog tree expansion

Abstract

A method for generating dialogs for learning a dialog policy includes, for each of at least one scenario, in which annotators in a pool of annotators serve as virtual agents and users, generating a respective dialog tree in which each path through the tree corresponds to a dialog and nodes of the tree correspond to dialog acts provided by the annotators. The generation includes computing a measure of uncertainty for nodes in the dialog tree, identifying a next node to be annotated, based on the measure of uncertainty, selecting an annotator from the pool to provide an annotation for the next node, receiving an annotation from the selected annotator for the next node, and generating a new node of the dialog tree based on the received annotation. A corpus of dialogs is generated from the dialog tree.

Description

BACKGROUND[0001]The exemplary embodiment relates to the field of dialog systems and finds particular application in connection with a system and method for expanding a dialog tree for learning a dialog system.[0002]Spoken dialog systems (SDS) have recently become widely used in human-computer interfaces, especially for access to various public information systems. These system use a virtual agent to conduct a dialogue with a client using a dialog manager to predict the next utterance of the agent. Despite their widespread use, there are still a number of challenges that have slowed their development. Among these are the time and cost of building such systems and the lack of expertise and training data. Various methods for developing SDS have been proposed, including statistical learning approaches, such as Reinforcement Learning (RL) (R. S. Sutton, et al., “Reinforcement Learning: An Introduction,” MIT Press, 1998), and rule-based hand-coded methods (Steve J. Young, “Using POMDPs fo...

AI Technical Summary

Benefits of technology

This patent describes a method and system for generating dialogs for learning a dialog policy. The method involves annotators in a pool of annotators serving as virtual agents and users, and generating a dialog tree for each scenario. The uncertainties of nodes in the tree are measured, and the next node to be annotated is identified based on this measure. Annotators are selected from the pool to provide annotations for the next node, and a new node is generated based on the received annotation. This process is repeated iteratively until a corpus of dialogs is generated. The system includes memory for storing the dialog trees, a tree update component for updating the dialog trees, and a reactive tree expansion component for progressively expanding the dialog trees. The main technical effect of this patent is the efficient generation of a large number of dialogs for learning a dialog policy.

Problems solved by technology

Despite their widespread use, there are still a number of challenges that have slowed their development.

Among these are the time and cost of building such systems and the lack of expertise and training data.

However, a problem with statistical approaches is that they rely on the availability of a large quantity of data.

A problem with this approach is that the interaction models built off-line using handcrafted conversational models are often poor approximations of the way humans actually interact with computers.

85-86, 1997), none of these tools supports application of the Wizard of Oz technique.

A drawback of these systems, however, is that they only allow finite-state dialogue modeling, which is restricted in its expressiveness.

However, no mechanism of active or reactive learning of dialog-tree expansion in the context of Wizard of Oz experiments has been suggested.

Images