HMD Camera Control via Eye Tracking Coordinate Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users wearing head-mounted display devices (HMDs) face difficulties in performing camera settings due to reduced visibility of the real space when trying to operate a digital camera, as they need to remove the HMD to access camera controls.
Innovation Solution
An electronic device with a processor and memory that transforms user-designated position coordinates in a display image into corresponding coordinates in the captured image, allowing for easy camera setting control while wearing HMDs by reversing the transformation processing applied to the live view image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a user wears an HMD to view a live view video, then the user can see the real space through the live view video, but the visibility of the real space is reduced and the user cannot easily perform camera setting
Solution Approach 1:
The patent introduces an intermediary mechanism (eye tracking system and coordinate transformation algorithm) that mediates between the user's gaze direction and the camera control system. The HMD detects the user's eye position and transforms display coordinates to camera coordinates, enabling indirect control of camera settings without requiring the user to physically interact with camera buttons or remove the HMD.
Solution Approach 2:
The patent replaces the mechanical interaction system (physically touching camera buttons or dials) with an optical/electronic system (eye tracking and coordinate transformation). The user's gaze direction is detected optically and converted into control signals for camera operations, eliminating the need for mechanical contact with camera controls while wearing the HMD.
2Ease of operation
If a user removes the HMD to operate the camera, then the user can access camera controls, but the user cannot see the real space through the live view video
Solution Approach 1:
The HMD acts as an intermediary device that provides both the live view video display and camera control interface simultaneously. The eye tracking system and coordinate transformation algorithm enable the HMD to translate the user's gaze into camera control commands, eliminating the need to remove the HMD for operation.
Solution Approach 2:
The HMD is designed to perform multiple functions simultaneously: displaying the live view video, tracking eye position, and controlling camera operations. This multi-functionality allows the user to view the live view video and operate camera controls through the same device without needing to switch between separate devices or remove the HMD.
3Loss of information
If the live view video is displayed on the HMD, then the user can view the captured image, but the visibility of the real space is reduced making camera setting difficult
Solution Approach 1:
The patent replaces mechanical camera control operations with an eye-tracking-based electronic control system. The HMD detects eye position and uses coordinate transformation to convert gaze direction into camera control commands, allowing users to adjust camera settings through gaze rather than physical interaction with camera buttons.
Solution Approach 2:
The coordinate transformation algorithm acts as an intermediary that translates between the HMD's display coordinate system and the camera's control coordinate system. This intermediary layer enables seamless communication between the display system and camera controls, allowing gaze-based operation while maintaining full camera functionality.
Data Source
AI summary
An electronic device according to the present invention includes: a processor; and a memory storing a program which, when executed by the processor, causes the electronic device to acquire position coordinates designated by a user in a display image obtained by performing first transformation processing on a captured image obtained by an imaging apparatus; transform the position coordinates in the display image into position coordinates in the captured image by a second transformation processing opposite to the first transformation processing; and perform control to perform a setting for the imaging apparatus, on a basis of the position coordinates in the captured image.