Automated Assistant Unstructured Utterance Prediction via Dynamic Bayesian Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Voice assistants struggle to perform intended actions for unstructured utterances, leading to a degraded user experience due to reliance on default responses and lack of adaptation to user-specific commands, raising privacy concerns and inefficiencies in training models.
Innovation Solution
A method and system utilizing an automatic speech recognition (ASR) device and dynamic Bayesian networks (DBN) to extract voice features, identify unstructured utterances, and compute probable actions based on context and similarity parameters, recommending actions to users with posterior probability calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If voice assistants are trained with structured utterances having action, title/subject and entities, then the voice assistant can reliably trigger intended tasks, but the voice assistant cannot perform user intended actions for unstructured utterances
Solution Approach 1:
The system dynamically adapts its behavior based on utterance structure. For structured utterances, it follows predefined action-resolution paths; for unstructured utterances, it activates alternative processing modes including search and recommendation mechanisms, allowing the system to be both reliable for trained commands and adaptable to novel inputs
Solution Approach 2:
The system changes its operational parameters based on utterance characteristics. When detecting unstructured utterances, it transitions from strict action-resolution mode to a more flexible mode that incorporates similarity-based matching, search operations, and probabilistic action recommendations, thereby handling both structured and unstructured inputs effectively
2Ease of operation
If the voice assistant provides default actions for unstructured utterances, then the system maintains simple operation, but the user experience degrades with responses like 'I didn't understand' or search results
Solution Approach 1:
The system provides feedback to users by recommending multiple probable actions with confidence scores for unstructured utterances. This allows users to see what the system understands and select from suggested actions, rather than receiving generic 'I didn't understand' responses, thereby reducing information loss while maintaining operational simplicity
Solution Approach 2:
The system introduces an intermediary recommendation layer between utterance recognition and action execution. For unstructured utterances, it generates and presents multiple probable actions as intermediates, allowing users to bridge the gap between their intent and system capabilities without requiring complex user input
3Adaptability or versatility
If the system collects individual user actions for model retraining, then the model can adapt to new utterances, but major privacy concerns arise from centralized collection
Solution Approach 1:
The system performs local adaptation at each device by processing unstructured utterances and generating probable actions locally without requiring centralized collection of user data. The DBN model operates autonomously on-device, enabling the system to adapt to user-specific commands while maintaining user privacy and avoiding centralized data aggregation
4Device complexity
If existing systems merely consider acoustic features and previous user behavior, then the system operates with simple processing, but it provides poor experience for novice users who need generalization across users
Solution Approach 1:
The system implements a universal DBN-based action recommendation mechanism that serves all users regardless of their familiarity with the assistant. The same core processing pipeline handles both novice and experienced users, providing generalized adaptation capabilities without requiring complex user-specific configurations or increasing overall system complexity
Data Source
AI summary
A method to adaptively predict non-default actions against unstructured utterances by an automated assistant operating in a computing-system is provided. The method includes extracting voice-features based on receiving an input utterance from at-least one speaker by an automatic speech recognition (ASR) device, identifying the input utterance as an unstructured utterance based on the extracted voice-features and a mapping between the input utterance with one or more default actions as drawn by the ASR, obtaining at least one probable action to be performed in response to the unstructured utterance through a dynamic bayesian network (DBN). The method further includes providing the at least one probable action obtained by the DBN to the speaker in an order of the posterior probability with respect to each action.