Wearable Display Contextual Object Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Industrial users face inefficiencies in accessing and navigating complex technical documentation and multiple interrelated applications while performing maintenance tasks, as traditional methods require manual navigation and hand-use, limiting hands-free operation and workflow efficiency.
Innovation Solution
A wearable device with a head-mounted display (HMD) that uses optical input, location sensors, and voice commands to provide context-aware navigation through digital tags, allowing users to access specific content and applications hands-free, enabling efficient interaction with non-human-machine interface devices via an augmented reality dashboard.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If manual navigation or touch-based systems are used to access technical documentation, then users can find required content, but it requires hand usage and stops workflow, reducing productivity
Solution Approach 1:
The patent replaces manual navigation and touch-based interaction with voice command recognition. Users can access technical documentation, navigate through content, and interact with applications using spoken commands while keeping hands free, thereby maintaining workflow continuity without sacrificing information access capability
Solution Approach 2:
The system introduces an augmented reality interface as an intermediary between the user and technical documentation. This AR interface overlays contextual information and navigation options in the user's field of view, allowing hands-free access to documentation while maintaining awareness of the physical workspace and equipment
2Adaptability or versatility
If complex workflows require multiple interrelated applications, then comprehensive functionality is achieved, but navigation becomes complicated and time-consuming
Solution Approach 1:
The augmented reality interface provides context-aware navigation by displaying application-specific controls and information tailored to the current workflow stage and equipment being viewed. Each region of the AR display contains relevant navigation options for that specific context, allowing users to access multiple applications through localized, situation-relevant commands rather than navigating through a complex global menu structure
Solution Approach 2:
The system pre-loads and pre-organizes multiple interrelated applications and their associated technical documentation based on the detected equipment context. When a user approaches or views specific equipment, the system proactively prepares and displays the relevant applications and navigation paths in the AR interface, eliminating the need for users to manually search or navigate through unrelated applications
3Loss of information
If hierarchical data streams are stored in traditional forms, then comprehensive data organization is achieved, but management becomes unwieldy on mobile devices
Solution Approach 1:
The patent transforms traditional hierarchical data organization into a spatially distributed augmented reality interface. Instead of navigating through nested menus and folders on a flat mobile screen, the system maps hierarchical data relationships into three-dimensional space around the user's field of view. Parent-child relationships and data hierarchies are visualized as spatial arrangements of virtual objects and panels that users can navigate by moving their head or using voice commands, converting complex hierarchical navigation into intuitive spatial exploration
Data Source
Figure 1
Figure 2
Figure 3A~3C
AI summary
Systems and methods provide for object interaction based on an identified context. A wearable device, such as a head-mounted display device (HMD), is initiated. The HMD can obtain data via one or more sensors that corresponds to one or more non-human- machine interface (HMI) devices or objects. Based on the obtained data, the HMD can recognize and identify the non-HMI devices within an optically-encoded field of view of the HMD. The non-HMI devices can be viewed through the HMD along with virtual objects or identifiers associated therewith, and the virtual objects or identifiers can reference the viewed non-HMI devices. One or more commands associated with the virtual objects or indicators can then be received by the HMD to enable interaction with the viewed non-HMI devices. Based on the commands information can be provided for display or audio output.