Dialogue Data Collection via Template Paraphrasing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Designing fluent conversational systems is challenging due to the complexities of natural language and the difficulty in analyzing various user sentences and system actions, with existing methods like Wizard of Oz systems being expensive and inefficient, and current chatbots not being successful as conversational system progenitors.
Innovation Solution
A data collection system that uses crowd workers to identify sensible dialogue paths and perform context-dependent paraphrasing, generating annotated training data for dialogue systems, including semantic parsers, dialogue policies, and generation systems, based on a database schema.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a Wizard of Oz system is used to create a conversational system model, then the system can be trained with human-like dialogue data, but the development cost and time increase significantly
Solution Approach 1:
The patent uses synthetic dialogue data generated by template-based systems as copies of real human conversations to train the conversational system, eliminating the need for expensive and time-consuming Wizard of Oz experiments while maintaining training data quality
Solution Approach 2:
The patent performs preliminary actions by pre-defining dialogue templates and schemas before actual system deployment, allowing the system to generate training data automatically without requiring manual human interaction during the training phase
2Loss of information
If a Wizard of Oz system is used to create a conversational system, then dialogue data can be collected, but the system is constrained to human operator utterances which limits natural language coverage
Solution Approach 1:
The patent creates a universal template-based framework that can generate diverse dialogue patterns across multiple domains and contexts, allowing the system to handle various language styles and structures without being limited to specific human operator preferences
Solution Approach 2:
The patent uses parameterized templates with variables and placeholders that can be instantiated with different values to generate diverse utterances, enabling the system to produce a wide range of natural language variations while maintaining grammatical correctness
3Device complexity
If a conversational system is created from scratch by engineering natural language interfaces, then the system structure can be controlled, but the complexity of covering all potential interactions increases
Solution Approach 1:
The patent segments the conversational system into modular components including dialogue templates, schemas, and parameter definitions, allowing each component to be independently developed and maintained while collectively providing comprehensive interaction coverage
Solution Approach 2:
The patent introduces templates as an intermediary layer between the system's internal logic and natural language expressions, simplifying the mapping between structured data and diverse language forms while maintaining control over system behavior
4Productivity
If chatbots are used as progenitors of conversational systems, then development can start quickly, but they have not been shown to be successful
Solution Approach 1:
The patent performs preliminary actions by pre-defining dialogue templates and schemas that encode domain knowledge and conversation patterns, allowing the system to achieve high effectiveness from the start without requiring iterative learning from basic chatbot interactions
Data Source
AI summary
A data collection system is based on a general set of dialogue acts which are derived from a database schema. Crowd workers perform two types of tasks: (i) identification of sensical dialogue paths and (ii) performing context-dependent paraphrasing of these dialogue paths into real dialogues. The end output of the system is a set of training examples of real dialogues which have been annotated with their logical forms. This data can be used to train all three components of the dialogue system: (i) the semantic parser for understanding context-dependent utterances, (ii) the dialogue policy for generating new dialogue acts given the current state, and (iii) the generation system for both deciding what to say and how to render it in natural language.