HMD Voice Motion Interface Emulating Touch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile display devices, such as HMDs and smartphones, face challenges in interacting with web-based applications optimized for desktops, as they lack mechanisms for conventional touch or tactile input, leading to difficulties in user interaction and inefficient UI experiences, especially for field workers who need to access complex software applications on the go.
Innovation Solution
An intelligent integration module that enables voice-driven and motion-driven user interactions, allowing users to interact with web content on mobile devices through voice commands and head motions, emulating touch inputs when necessary, to provide a hands-free and intuitive user experience without modifying existing software applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If web-based applications are optimized for desktop displays, then the application functionality and information completeness are improved, but the ease of operation and user interaction quality deteriorate on mobile devices
Solution Approach 1:
The patent transitions from 2D touch-based interaction to 3D spatial interaction by utilizing head-mounted display capabilities. Users interact with web content through head movements and voice commands in a three-dimensional space, allowing full access to desktop-optimized applications without physical contact with the device.
Solution Approach 2:
The patent replaces mechanical touch input mechanisms with voice-based and motion-based interaction systems. Instead of requiring physical touch or mouse clicks, the system interprets voice commands and head movements to control web application interactions, enabling hands-free operation while maintaining full application functionality.
2Ease of operation
If software applications are rewritten for mobile devices, then the ease of operation is improved, but the device complexity and development resources increase
Solution Approach 1:
The patent creates a universal interaction layer that works across desktop and mobile platforms. The same web application can be accessed on mobile devices through the head-mounted display interface without requiring separate mobile versions, as the system adapts the interaction method rather than the application itself.
Solution Approach 2:
The patent introduces an intermediary system between the user and the web application that translates head movements and voice commands into standard web interaction commands. This intermediary layer handles the complexity of mobile interaction without requiring modifications to the underlying web application code.
3Ease of operation
If conventional touch input mechanisms are used on mobile devices, then the ease of operation is improved, but the adaptability to hands-free operation deteriorates
Solution Approach 1:
The patent implements dynamic interaction methods that adapt to the user's needs and environment. The system can switch between voice-based commands, head movement tracking, and other motion-based interactions depending on the situation, providing hands-free capability when needed while maintaining flexibility for other interaction modes.
Data Source
AI summary
Systems and methods are provided for implementing an interactive application, which incorporates voice-driven and motion-driven interactions with media content in a display device. An application instance can be initialized for interacting with media content output to a display device, such as an head mounted display (HMD). Then, a determination whether a received user interaction is interpretable into an interactive command defined by an operating system (OS) is performed. If the OS can interpret the user interaction, the interactive command can be executed with actions generated by the OS. Alternatively, an emulation of the interactive command may be executed, when the user interaction cannot be interpreted by the OS. Subsequently, the media content is presented within a user interface based on the interactive command. For example, user interaction can be the head movement of a user that is interpreted into a command that controls presentation of web content in the HMD.