Head-Mounted Display Hand Tracking via Spatial Image Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
See-through image display apparatuses have poor operability due to user difficulty in operating with their head covered by the screen, as existing techniques fail to improve user interaction effectively.
Innovation Solution
An image processing apparatus that generates and displays images based on input images captured by a camera near the user's viewpoint, calculating corrections for display position and size adjustments to align virtual and real-world interactions, allowing for improved user interface operations through semi-transparent displays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a touch interface or button interface is used in a see-through image display apparatus, then the device can display images, but the user cannot operate effectively with their head covered by the screen
Solution Approach 1:
The patent replaces traditional mechanical touch interfaces and button interfaces with a camera-based vision recognition system. The imaging unit captures images of the user's hands and operations, and the processing unit recognizes these operations to control the display apparatus, eliminating the need for physical contact with the device.
Solution Approach 2:
The patent introduces an imaging unit and image processing system as an intermediary between the user and the display apparatus. Instead of direct hand-to-screen contact, the user's hand movements are captured by the camera and translated into control commands through image recognition, creating an indirect but more flexible interaction mode.
2Measurement precision
If the user operates the device with head covered by the screen, then the display can be viewed, but the user's hand movements cannot be accurately detected
Solution Approach 1:
The patent transitions from two-dimensional screen-based touch detection to three-dimensional spatial detection using the camera. By capturing depth information and spatial position of hand movements, the system can accurately detect operations even when the user's head is positioned close to or covering part of the screen area.
Solution Approach 2:
The system performs preliminary image capture and processing to establish a reference frame and detect hand positions before the user actually performs operations. This allows the system to pre-establish the detection zone and coordinate system, ensuring accurate tracking of hand movements from the start of interaction.
Data Source
AI summary
The present technique relates to an image processing apparatus, method, and program which can improve operability. A short distance image separation unit generates a short distance image indicating a short distance object located at the side of an imaging unit, on the basis of a clipped image which is clipped from an input image captured by the imaging unit, and a distance image indicating the distance to an object in each region of the clipped image. A contour extraction unit extracts, as an adjustment image, a contour of the short distance image, and a display image generation unit generates an output image on the basis of the adjustment image and displays the output image on an image display unit. The user moves his/her hand while viewing the output image displayed on the image display unit so that his/her hand overlaps with the region of the hand in the output image. An XY calculation unit calculates the amount of correction of the display position of the image on the basis of adjustment images of different time points. The technique may be applied to a head mount display.