Mirror Display GUI Control Using Gaze-Based Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing display apparatuses with mirror functions lack the ability to seamlessly integrate user interaction and object recognition to provide dynamic and context-aware graphical user interfaces.
Innovation Solution
An electronic apparatus with a mirror display and sensor that identifies user commands and gaze information to display a GUI on the mirror based on recognized objects, using ultra-wideband signals for precise positioning and communication with external devices to control functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a mirror display is used to provide both mirror and display functions, then the versatility of the device is improved, but the ability to provide dynamic and context-aware GUIs is insufficient
Solution Approach 1:
The system continuously monitors user gaze information through sensors and uses this feedback to dynamically adjust the displayed content and position on the mirror display. The mirror display provides feedback about the reflected object to the processor, which then generates appropriate GUI elements based on the detected context, creating a closed-loop system that adapts to user needs in real-time.
Solution Approach 2:
The system performs preliminary actions by pre-identifying objects in the reflected scene using gaze information before the user actually interacts with them. The processor prepares and displays relevant GUI elements in advance based on the detected object and user attention, so that when the user looks at or touches the mirror display, the appropriate controls and information are already available.
2Adaptability or versatility
If object recognition and GUI display are integrated on the mirror display, then the functionality is improved, but the device complexity increases
Solution Approach 1:
The mirror display serves multiple functions simultaneously: it acts as a reflective surface for viewing objects, a display surface for showing GUI elements, and an interaction interface for touch input. The processor performs multiple tasks including object recognition, gaze tracking, GUI generation, and touch detection, all through a single integrated system rather than separate dedicated components.
Solution Approach 2:
The system uses the mirror display's own reflective property to identify objects in the scene, turning the display's primary function into a sensing mechanism. The gaze sensor uses the reflected light from objects to determine their position and characteristics, allowing the system to self-identify objects without requiring separate cameras or sensors pointing at the scene.
3Measurement precision
If precise positioning of GUI elements is achieved using gaze information, then the measurement precision is improved, but the difficulty of detecting and measuring increases
Solution Approach 1:
The mirror display acts as an intermediary between the physical object and the user's gaze. Instead of directly measuring the user's eye position and calculating object location, the system measures the gaze position on the mirror display surface, which directly corresponds to the reflected object's position. This intermediary approach simplifies the measurement process while maintaining high precision.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
An electronic apparatus including a mirror display; a sensor; and at least one processor configured to identify a user command for selecting an area of the mirror display, acquire user gaze information through the sensor, identify an object in a predetermined space and reflected in the selected area of the mirror display based on the acquired user gaze information, identify a graphic user interface (GUI) to be displayed on the mirror display based on information on the identified object, identify a position on the mirror display to display the identified GUI based on the user gaze information, and control the mirror display to display the identified GUI on the identified position.