Real-Time Conversational Agent NLU via Crowd Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current real-time automated messaging interfaces face challenges in accurately understanding user intent due to the infinite variations of human requests and responses, leading to suboptimal performance and the need for extensive predefined scripts, which are impractical to cover all possible scenarios.
Innovation Solution
A method utilizing a real-time crowd-assisted natural language understanding process that reduces possible user intents and data elements to a high-certainty set using statistical and pattern matching methods, then sends these to crowd participants for voting, allowing for human-inferred insights and the rejection of options to improve accuracy and response efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If predefined scripts and statistical analysis are used to identify user intent, then the system can provide automated responses, but the accuracy of understanding user intent deteriorates due to infinite variations of human requests
Solution Approach 1:
The patent introduces a crowd-sourced verification layer as an intermediary between the automated messaging interface and the predefined scripts. Human crowd workers validate and correct the system's interpretation of user intent, bridging the gap between automated processing and accurate understanding. This mediator resolves the contradiction by allowing automation to operate while maintaining high accuracy through human verification of ambiguous cases.
2Adaptability or versatility
If more predefined scripts are created to cover all possible human responses, then the coverage of user intents improves, but the device complexity increases significantly
Solution Approach 1:
The system employs crowd workers to dynamically create and refine response scripts based on actual user interactions. Instead of requiring the system to pre-define all possible scripts, the crowd service enables the system to self-improve and adapt its script library organically through real-world usage data and human feedback, reducing the burden of manual script creation while expanding coverage.
Solution Approach 2:
The patent uses statistical analysis and pattern matching to pre-process and categorize user intents before they reach the script matching stage. This preliminary action groups similar requests together, allowing the system to handle variations of human requests through a smaller set of generalized scripts rather than requiring separate scripts for every possible variation.
3Measurement precision
If crowd participants are used to vote on user intents in real-time, then the accuracy of natural language understanding improves, but the response time increases due to the voting process
Solution Approach 1:
The system implements a confidence threshold mechanism where crowd verification is only applied to cases that exceed a certain ambiguity threshold. For clear, unambiguous user intents, the system responds immediately using its automated analysis without invoking the crowd. This partial application of crowd verification maintains real-time responsiveness for most cases while still improving accuracy for ambiguous situations.
Data Source
AI summary
Systems, methods and computer-readable storage media for natural language understanding in combination with real-time automated humanized verification in conversation agent messaging are described.