Native GUI Voice Interaction via Structured Command Translation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice-based interaction systems for executable applications on mobile devices lack seamless integration with native graphical user interfaces, failing to efficiently translate voice inputs into application-specific commands due to limitations in speech recognition and natural language processing.
Innovation Solution
A system that embeds voice-user interface components within native graphical user interfaces, utilizing speech recognition, natural language processing, and semantic parsing to translate voice inputs into structured commands for executable applications, enabling seamless voice-based interaction and control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If voice-based input systems are implemented for executable applications, then user interaction capability is improved, but integration with native graphical user interfaces and translation efficiency into application-specific commands deteriorates
Solution Approach 1:
The patent introduces a voice service provider application as an intermediary layer between the user's voice input and the executable application. This mediator captures voice input, translates it into structured data through speech recognition and natural language processing, and delivers the translated commands to the application through a programmatic interface, thereby resolving the integration difficulty between voice-based systems and native GUIs
Solution Approach 2:
The system divides the voice interaction process into distinct functional modules: a voice service provider application handling speech recognition and natural language processing, an executable application receiving structured commands, and a programmatic interface facilitating communication. This segmentation allows each component to specialize in specific tasks, improving overall translation efficiency while maintaining ease of operation
2Measurement precision
If speech recognition and natural language processing are enhanced, then translation accuracy of voice inputs is improved, but system complexity deteriorates
Solution Approach 1:
By positioning the voice service provider application as an intermediary, the patent concentrates complex speech recognition and natural language processing functionalities in a dedicated component. This allows the main executable application to remain relatively simple while still achieving high translation accuracy through the specialized mediator's processing capabilities
Solution Approach 2:
The patent extracts complex speech recognition and natural language processing functions from the main executable application and places them in a separate voice service provider application. This extraction reduces the complexity of the primary application while maintaining high translation accuracy through the specialized external processing system
3Ease of operation
If voice-user interface components are embedded in native GUI, then user experience is improved, but development complexity deteriorates
Solution Approach 1:
The voice service provider application serves multiple functions: it acts as a microphone icon in the GUI, processes speech recognition, performs natural language processing, and communicates with applications through standardized programmatic interfaces. This multi-functionality allows a single component to deliver enhanced user experience while avoiding the need for separate development of multiple voice-related features
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
The disclosed embodiments include computerized methods, systems, and devices, including computer programs encoded on a computer storage medium, for integrating voice-based interaction and control into a native graphical user interface (GUI) of an executed application. For example, a communications device may receive audio data. The communications device may provide structured data to the executed application through a programmatic interface, and the executed application may perform the one or more operations in accordance with the structured data. The communications device may generate data indicative of an output of the one or more operations performed by the executed application, and may present at least a portion of the generated output data to a user through a corresponding interface.