Head Gesture Display Control via 3D Pose Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture-based solutions for controlling display devices are limited by the need for hand movements, which can be restrictive for individuals with disabilities and in environments where hand motion is restricted, such as public places, and do not allow access to all areas of the screen.
Innovation Solution
A method and system that captures and processes head gestures to estimate a 3D head pose, mapping it to a cursor on a display screen, enabling control of the user interface through subtle head motions, allowing access to any location on the screen without the need for hand movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If hand gestures are used for display device control, then the interaction is more natural compared to traditional remote controls, but the user's hands must be free and movable which restricts accessibility for users with disabilities and in environments where hand movement is limited
Solution Approach 1:
The patent changes the control parameter from hand gestures to head gestures. By detecting head pose parameters (pitch, yaw, roll angles) and mapping them to cursor position on the display screen, the system enables control through head movement instead of hand movement, making it accessible to users with hand disabilities while maintaining natural interaction principles
Solution Approach 2:
The patent replaces the mechanical hand gesture recognition system with a head pose detection system using computer vision and image processing. The mechanical movement of hands is substituted by detecting the orientation and position of the head through facial landmark analysis and 3D pose estimation
2Ease of operation
If hand gestures are used to perform selections on an interactive screen, then the user can control the display device, but the limited range of hand motion impedes the user from reaching every location on the screen
Solution Approach 1:
The patent transitions from 2D hand movement on a limited plane to 3D head pose control. By utilizing three rotational degrees of freedom (pitch, yaw, roll) of the head, the system maps a larger volumetric control space to the 2D screen area, enabling access to all screen locations including corners that would be difficult to reach with hand gestures
3Ease of operation
If hand movements are used for display device control in public places, then the user can interact with the display, but movement of the hands is restricted in crowded public places such as airports, railway stations, and shopping centres
Solution Approach 1:
The patent changes the control modality from hand-based to head-based gestures, allowing users to interact with display devices in environments where hand movement is restricted. Head gestures can be performed subtly and do not require the same physical space or freedom of movement as hand gestures, making the system adaptable to crowded public places
Data Source
AI summary
A method for remotely controlling a computing device comprises repeatedly capturing an image from a video frame, detecting a human face in the captured image, matching the detected human face to a previously detected human face, extracting facial landmarks from the matched detected human face, estimating a 3D head pose of the matched detected human face based on the extracted facial landmarks, the 3D head pose being represented in an egocentric coordinate system by a 3D pose vector which is directed from the human face, the 3D pose vector being free to rotate around x-, y- and z-axes of the egocentric coordinate system using respective rotation matrices and free to translate along these x-, y- and z-axes using a translation vector, and controlling a user interface on a display screen of the computing device according to the estimated 3D head pose.