Augmented Reality HMI for Industrial Equipment Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Field operators in industrial settings face challenges due to the inconsistency and complexity of various user interfaces for different equipment, requiring them to switch between multiple systems and access disparate documentation, which disrupts their workflow and makes it difficult to monitor and control industrial processes efficiently.
Innovation Solution
The implementation of an augmented reality (AR) system that generates virtual information overlaid on real-world images, allowing field operators to interact with and control industrial equipment using a virtual control panel displayed on wearable devices, providing real-time data and documentation directly at the equipment location, and enabling remote expert interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple separate user interfaces are used for different equipment, then each interface can be optimized for its specific device, but the operator must switch between multiple interfaces which reduces efficiency and increases complexity
Solution Approach 1:
The patent implements a universal augmented reality interface that can display and control multiple different equipment types simultaneously. The HMI system provides a standardized interaction model that adapts to various equipment (robots, conveyors, ovens, etc.) through a common graphical interface, eliminating the need for operators to switch between device-specific interfaces while maintaining optimized control capabilities for each equipment type
2Loss of information
If documentation is stored in separate electronic systems at different locations, then comprehensive information can be maintained, but operators must interrupt tasks to access documentation which reduces productivity
Solution Approach 1:
The patent merges the HMI interface and documentation access into a single integrated augmented reality system. Operators can access equipment manuals, maintenance history, and operational documentation directly through the AR glasses without switching devices or locations. The system combines real-time equipment data, control functions, and documentation in one unified interface that follows the operator throughout the workspace
Solution Approach 2:
The system provides self-service documentation delivery by automatically presenting relevant information based on the operator's location and current task. The AR interface detects which equipment the operator is viewing and automatically displays appropriate documentation and guidance, eliminating the need for operators to actively search for information or plan information gathering in advance
3Ease of operation
If standardized HMI interface is implemented across all equipment, then operator training is simplified and interface consistency is improved, but equipment-specific customization options are reduced
Solution Approach 1:
The patent implements a dynamic HMI that adapts its display content and control options based on the specific equipment being viewed. The standardized AR interface framework automatically configures itself to show equipment-specific parameters, controls, and documentation relevant to each device type. The system transitions between different equipment contexts dynamically as the operator moves through the workspace, providing both consistency in interaction model and customization in displayed content
Data Source
AI summary
A method to generate technical information about a device or process by recognizing the identity of a device or process, retrieving a stored image dependent on the device, combining the stored virtual image together with an image of a real device object, displaying the combination on a display and providing a virtual control HMI means to the user for monitoring and/or controlling the device or process. A graphical user interface, computer program and a system are also described.