GUI Speechability Scoring for Voice Interface Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graphical user interfaces (GUIs) on electronic devices, particularly on small-screen devices like smartphones, face challenges in user interaction due to limited input and output options, where the complexity of voice user interfaces (VUIs) can hinder recognition of speech interactions, making it difficult to determine whether adding a VUI would enhance user efficiency and experience.
Innovation Solution
A method that analyzes GUI screens to determine the quantity of text, image, audio, and video objects, assigns a speechability score to each screen, and identifies a threshold to determine which screens would benefit from a voice user interface (VUI), thereby enabling targeted implementation to improve user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a voice user interface (VUI) is added to handle quick and routine interactions, then user efficiency and experience are improved, but the complexity of speech interactions increases making recognition more challenging
Solution Approach 1:
The system changes the parameter of interface modality by dynamically selecting between GUI and VUI based on the speechability score. When the score indicates suitability for voice interaction, the system transitions to VUI mode, leveraging the natural suitability of speech for quick interactions while avoiding the complexity of speech recognition for inappropriate tasks.
Solution Approach 2:
The speechability scoring system automatically evaluates each GUI screen to determine its suitability for VUI without requiring manual configuration. The system self-determines which screens benefit from voice interaction based on objective criteria (number of text fields, buttons, etc.), enabling automatic, context-appropriate modality selection.
2Ease of operation
If speech-based interface is implemented on small-screen devices, then hands-free interaction capability is improved, but the challenge of providing input and output through limited display area increases
Solution Approach 1:
The system implements a universal interface framework that can operate in multiple modes (GUI and VUI) depending on the task requirements. By making the interface modality universal rather than fixed, the system can provide hands-free operation when appropriate while maintaining traditional graphical interaction when more complex input/output is needed, thus solving the limitation of small display areas.
3Productivity
If VUI is added to GUI, then user experience for routine interactions is improved, but the difficulty of determining which screens benefit from VUI increases
Solution Approach 1:
The system replaces manual evaluation of screen speechability with an automated computational scoring mechanism. The speechability score algorithm objectively evaluates each screen's characteristics (counting text fields, buttons, and other UI elements) to determine VUI suitability, eliminating the difficulty of manual determination and providing consistent, data-driven decisions.
Data Source
AI summary
Techniques are disclosed for identifying which graphical user interface (GUI) screens of an application that is under development would benefit from a voice user interface (VUI). A GUI screen parser analyzes to determine the GUI objects within GUI screens of the application. The parser assigns a speechability score to each analyzed GUI screen. Those GUI screens that have a higher speechability score than a predetermined speechability threshold are indicated as GUI screens that would benefit (e.g., the user experience in interacting with those GUI screens would increase, the number of GUI screens displayed would be reduced, or the like) with the addition of a VUI.