Simultaneous Touch and Voice Control for Mobile Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mobile devices lack the ability to seamlessly integrate simultaneous or near-simultaneous voice and touch gesture controls, limiting the fluidity and intuitiveness of user interactions with applications.
Innovation Solution
Combining touch gesture functionality with speech recognition and natural language processing to enable mobile devices to receive and process both speech and touch inputs simultaneously, allowing for new actions and functionalities that can be performed in a single step.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If speech recognition functionality is added to mobile devices, then text input efficiency is improved, but device complexity increases
Solution Approach 1:
The patent combines speech recognition functionality with the existing touch screen interface into a unified control system. The speech-to-text converter and natural language processor are integrated with the touch screen controller, allowing users to switch between voice and touch inputs seamlessly within the same application interface, thereby improving text input efficiency without significantly increasing perceived device complexity.
Solution Approach 2:
The mobile device is designed to perform multiple functions through a single integrated system. The touch screen serves both as a visual display and as an input interface that can work in conjunction with speech recognition. The same processor handles both touch gestures and speech-to-text conversion, making the device more versatile without requiring separate dedicated hardware for each function.
2Ease of operation
If simultaneous touch and voice control is implemented, then user interaction fluidity is improved, but processing complexity increases
Solution Approach 1:
The system prepares for simultaneous touch and voice control by pre-configuring the touch screen interface to recognize specific gesture patterns that trigger voice input modes. The speech-to-text converter is pre-loaded and ready to process speech immediately when activated by a touch gesture, reducing the processing complexity of handling both inputs simultaneously by having the system already in the appropriate state.
Solution Approach 2:
The natural language processor acts as an intermediary between the touch screen interface and the speech-to-text converter. It receives both touch gesture data and speech input, processes them together to determine user intent, and coordinates the execution of combined commands. This mediator simplifies the processing complexity by providing a centralized coordination point rather than requiring complex direct integration between all components.
3Loss of time
If speech-to-text conversion is added, then typing time is reduced, but energy consumption increases
Solution Approach 1:
The speech-to-text converter is activated selectively rather than continuously. It is triggered only when the user performs specific touch gestures or selects voice input mode, allowing the system to save energy by keeping the speech processing functions dormant when not needed. This partial activation approach reduces energy consumption while still providing the time-saving benefit when voice input is actually used.
Data Source
AI summary
For a mobile computing device, enhanced functionality may be provided by associating actions with combined speech and touch gestures. A touch gesture is received into the device and simultaneously or near-simultaneously speech is received into the device. The touch gesture and speech are processed to determine a result and the device performs an action based on the result. In particular embodiments, commands for a mapping application may be based on spoken search terms and geographic areas marked by touch gestures.