Contextual Utterance Recommendations for Conversational Visual Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face challenges in formulating natural language utterances for visual data analysis due to the need to understand data domain characteristics and patterns, and limitations in natural language understanding capabilities, leading to inefficient interactions with visual analytical systems.
Innovation Solution
The implementation of a mixed-initiative natural language interface, 'Snowy', which generates and recommends utterances based on data interestingness metrics and language pragmatics, providing contextual guidance and awareness of the system's interpretation capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users formulate natural language utterances directly without system guidance, then users maintain full control over query formulation, but users experience high cognitive burden and interaction inefficiency due to needing to understand data domain characteristics and system interpretation capabilities
Solution Approach 1:
The system performs preliminary analysis of the data domain and generates suggested utterances before the user formulates their query. The mixed-initiative NLI proactively provides context-aware utterance recommendations based on the current visual analysis state, allowing users to select from pre-generated options rather than formulating queries from scratch, thereby reducing both cognitive burden and time required
Solution Approach 2:
The system continuously monitors the visual analysis state and user interactions, then provides feedback in the form of suggested utterances that are contextually relevant to the current analysis stage. This feedback loop enables users to refine their queries based on system recommendations while maintaining alignment with their analytical goals, improving both ease of operation and time efficiency
2Loss of information
If the system provides comprehensive utterance recommendations, then users gain analytical guidance and system capability awareness, but the interface complexity increases
Solution Approach 1:
The system provides utterance recommendations with varying degrees of detail and specificity based on the local context of the visual analysis state. Rather than providing uniform comprehensive recommendations, the system adapts the granularity and type of suggestions to match the current analytical needs, presenting only the most relevant guidance at each stage to avoid overwhelming the user while maintaining information availability
Solution Approach 2:
The recommendation interface dynamically adjusts its behavior based on user interactions and the evolving visual analysis state. The system can transition between providing detailed guidance and more minimal suggestions, allowing the interface complexity to adapt to the user's expertise level and current analytical needs, thereby balancing information provision with interface simplicity
Data Source
AI summary
A computing device performs an automated analysis of a data source and generates one or more first natural language outputs in accordance with the automated analysis. The device displays the first natural language outputs in a graphical user interface and receives user selection of a natural language output of the first natural language outputs. In response to receiving the user selection, the device generates a first data visualization according to the automated analysis, one or more data fields of the data source specified in the natural language output, and/or analytical operations specified in the natural language output. The device displays the first data visualization in the graphical user interface. The device generates one or more second natural language outputs and displays the second natural language outputs in the graphical user interface.