Reactive learning for efficient dialog tree expansion

Abstract

A method for generating dialogs for learning a dialog policy includes, for each of at least one scenario, in which annotators in a pool of annotators serve as virtual agents and users, generating a respective dialog tree in which each path through the tree corresponds to a dialog and nodes of the tree correspond to dialog acts provided by the annotators. The generation includes computing a measure of uncertainty for nodes in the dialog tree, identifying a next node to be annotated, based on the measure of uncertainty, selecting an annotator from the pool to provide an annotation for the next node, receiving an annotation from the selected annotator for the next node, and generating a new node of the dialog tree based on the received annotation. A corpus of dialogs is generated from the dialog tree.

Description

BACKGROUND[0001]The exemplary embodiment relates to the field of dialog systems and finds particular application in connection with a system and method for expanding a dialog tree for learning a dialog system.[0002]Spoken dialog systems (SDS) have recently become widely used in human-computer interfaces, especially for access to various public information systems. These system use a virtual agent to conduct a dialogue with a client using a dialog manager to predict the next utterance of the agent. Despite their widespread use, there are still a number of challenges that have slowed their development. Among these are the time and cost of building such systems and the lack of expertise and training data. Various methods for developing SDS have been proposed, including statistical learning approaches, such as Reinforcement Learning (RL) (R. S. Sutton, et al., “Reinforcement Learning: An Introduction,” MIT Press, 1998), and rule-based hand-coded methods (Steve J. Young, “Using POMDPs fo...

AI Technical Summary

Problems solved by technology

Despite their widespread use, there are still a number of challenges that have slowed their development.

Among these are the time and cost of building such systems and the lack of expertise and training data.

However, a problem with statistical approaches is that they rely on the availability of a large quantity of data.

A problem with this approach is that the interaction models built off-line using handcrafted conversational models are often poor approximations of the way humans actually interact with computers.

85-86, 1997), none of these tools supports application of the Wizard of Oz technique.

A drawback of these systems, however, is that they only allow finite-state dialogue modeling, which is restricted in its expressiveness.

However, no mechanism of active or reactive learning of dialog-tree expansion in the context of Wizard of Oz experiments has been suggested.

Images