Dialogue Data Collection System with Automatic Achievement Determination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for collecting dialogue data, such as conventional worker interaction and crowdsourcing, face challenges in efficiency and quality due to worker errors, fraud, and unclear task completion, leading to decreased work efficiency and subpar data quality.
Innovation Solution
A dialogue data collection system comprising a task presenter, acquirer, response information presenter, achievement determiner, notifier, and collector, which presents tasks to workers, acquires and processes spoken phrases, determines task achievement based on predefined conditions, and notifies workers of completion, ensuring high-quality data collection without observer oversight.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If crowdsourcing is used to collect dialogue data, then collection cost and time are reduced, but data quality deteriorates due to worker errors and fraud
Solution Approach 1:
The system implements automatic feedback mechanisms where the dialogue system evaluates worker responses in real-time and provides guidance on task completion status. This closed-loop feedback ensures workers receive immediate correction if they make errors or attempt fraud, while maintaining the efficiency benefits of crowdsourcing by eliminating the need for human observers.
2Ease of operation
If workers determine task completion independently, then observer oversight is eliminated, but task completion accuracy deteriorates
Solution Approach 1:
The system enables workers to complete tasks autonomously through self-service interfaces, while the dialogue system automatically monitors and validates their work. The achievement determination unit autonomously evaluates whether tasks are completed based on predefined criteria, eliminating the need for external observers while maintaining high accuracy through automated assessment.
3Reliability
If automatic achievement determination is implemented, then data quality improves, but system complexity increases
Solution Approach 1:
The dialogue system performs multiple functions simultaneously: it collects dialogue data, evaluates worker responses, determines task achievement, and provides feedback to workers. This multi-functionality reduces overall system complexity by consolidating what would otherwise require separate specialized components into a single integrated dialogue management system.
Data Source
AI summary
According to an embodiment, a dialogue data collection system includes a presenter, a determiner, a notifier and a collector. The presenter receives response information that the dialogue management apparatus generates by advancing a dialogue status based on the spoken phrases, and presents the response information to the worker. The determiner compares the response information to one or more achievement conditions predetermined relative to the task, and determines achievement or non-achievement of the task based on whether or not the achievement conditions are satisfied. The notifier notifies the worker of achievement of the task. The collector collects dialogue data including the spoken phrase and the response information.