Voice User Interface Response Matching for Disfluent Speech
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Voice User Interfaces (VUIs) face challenges in effectively understanding and responding to users with speech disfluencies and low-resource languages, particularly Basic Emergent Users (BEUs), leading to inefficiencies in communication and usability.
Innovation Solution
An Information-entropy-based metric is employed to analyze spoken queries for disfluencies, pauses, and syllables, using a Large Language Model (LLM) to generate optimal text responses that match the user's communication style, ensuring the machine response is efficient and understandable.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If VUI uses standard speech recognition and response generation, then system complexity is low, but communication ability matches poorly with BEU users who have speech disfluencies
Solution Approach 1:
The system pre-computes entropy metrics for user queries and generates multiple candidate responses with varying levels of detail before user interaction. This preliminary preparation allows the system to quickly select the most appropriate response without complex real-time processing, thereby improving adaptability while managing system complexity
Solution Approach 2:
The VUI dynamically adjusts its response generation process based on computed entropy metrics of the user's query. The system varies the level of detail and structure in responses according to the measured communication characteristics, enabling adaptive matching with BEU users while maintaining manageable system complexity through conditional logic
2Loss of information
If VUI generates detailed responses, then information completeness is high, but usability for BEU users with limited communication ability decreases
Solution Approach 1:
The system applies different levels of response detail locally based on the specific entropy characteristics of each user query. Rather than using a uniform response style, the VUI tailors the information density and structural complexity to match the measured communication ability of the particular user interaction, thereby preserving information completeness while enhancing usability
Solution Approach 2:
The system changes key parameters of the response generation process, including sentence length, vocabulary complexity, and information density, based on the computed entropy metrics of the user's query. This dynamic parameter adjustment allows the VUI to maintain information completeness while adapting to the specific communication needs of BEU users
3Measurement precision
If VUI processes speech disfluencies and pauses in real-time, then communication accuracy improves, but processing time increases
Solution Approach 1:
The system performs entropy computation and communication ability assessment as preliminary processing steps before generating the actual response. By preparing these metrics in advance during the user's speech input phase, the system avoids time-consuming analysis during the response generation phase, thereby maintaining high communication accuracy while reducing overall processing time
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
A method and system for Information-entropy-based metric for usable machine response of a Voice User Interface (VUI) to match with communication ability of a speaker is disclosed. The metric disclosed herein dynamically, on the fly analyses every received query for disfluencies such as 'hmm' and 'aah,' hesitation leading to pauses in speech, repetition, and vocabulary etc., to determine the property of the query in terms of communication ability or entropy in the query. A Large language Model (LLM) responding to the query is configured to generate and select and optimal response to the query such that efficiency of expression of the response to efficiency of expression of the query is minimal. Unlike the VUI analysis in the art, the interaction design for the VUI disclosed herein understands the mental model of the user (speaker) and communicate the system's response to the user in the user's language and communication style.