Augmented Machine Interface With Synced Virtual Overlay Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current camera display systems for large mobile equipment provide limited views of the equipment's functionality and interaction with its environment, requiring operators to monitor both camera views and the equipment in real life while physically moving around.
Innovation Solution
An augmented user interface (AUI) system that combines real-time camera views with a dynamic virtual model of the machine, using sensors to track the movement of machine parts and overlay the virtual model onto the camera footage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If operators use camera display systems to view areas around the machine, then they can monitor distant areas, but they have limited views of the machine's functionality and interaction with its environment
Solution Approach 1:
The system merges camera views showing the machine environment with a virtual model of the machine to create an augmented view. This combination allows operators to simultaneously see both the external environment captured by cameras and the internal machine functionality represented by the virtual model, eliminating the need to switch between different display modes and providing comprehensive situational awareness in a single integrated display.
Solution Approach 2:
The system creates a virtual copy of the machine that mirrors the actual machine's state in real-time. This virtual model is generated from sensor data and displayed overlaid on camera footage, allowing operators to monitor machine functionality through the copy without needing to physically inspect or switch to separate displays showing machine internals.
2Ease of operation
If operators physically move around the machine to observe different portions, then they can view machine functionality, but they cannot simultaneously monitor camera views and machine operation from a single location
Solution Approach 1:
The system replaces the mechanical requirement for operator movement with an electronic/digital solution. Instead of physically moving to different locations on the machine to observe different portions, operators use sensors and virtual modeling to electronically replicate and display machine state data on fixed displays, eliminating the need for physical repositioning while maintaining comprehensive monitoring capability.
3Loss of information
If operators monitor both camera views and machine in real life, then they can gather comprehensive information, but they must divide attention between multiple sources
Solution Approach 1:
The system combines camera views and machine monitoring data into a single integrated augmented display. The virtual model is overlaid directly on the camera footage, allowing operators to view both environmental context and machine functionality simultaneously in one location, eliminating the need to divide attention between separate displays or physical observations.
Data Source
AI summary
A method and system for generating an augmented machine view. The augmented machine view may include a dynamic virtual machine model representing an actual working machine, overlayed with camera footage from around the working machine. The dynamic virtual machine model may be synched in time with the camera footage. The system for generating the augmented machine view may include cameras mounted on the actual machine and machine sensors monitoring the position and movements of individual components of the machine.