3D Display Eye Gaze and Hand Gesture Interaction System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human-machine interaction methods lack the ability to truly simulate natural interactive behaviors, resulting in a poor user experience due to inadequate simulation of gesture controls.
Innovation Solution
A method that utilizes a 3D display system with a sight tracking camera and a gesture detection camera to detect eye gaze and hand interactions, allowing for the rendering of high-definition visuals when the user's eyes and hand engage with virtual objects, while reducing bandwidth by rendering other areas in low definition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gesture control methods are used to control content on the display screen, then the interaction method is natural and comfortable, but the current human-machine interaction mode has poor user experience because it does not truly simulate people's interactive behaviors
Solution Approach 1:
The patent combines eye tracking technology with gesture recognition technology to create a unified interaction system. The eye tracking camera detects gaze direction while the gesture camera detects hand movements, and these two detection systems are merged to control the 3D display content, enabling a more complete simulation of natural human interaction behaviors that neither system could achieve alone
Solution Approach 2:
The patent introduces an interaction processing device as an intermediary that receives detection results from both the eye tracking camera and gesture camera, processes this information according to preset interaction logic, and generates control signals for the 3D display. This intermediary coordinates the two detection systems and translates natural human behaviors into meaningful control actions
2Manufacturing precision
If high definition rendering is applied to the entire 3D view, then the visual quality is high, but the display bandwidth requirement increases significantly
Solution Approach 1:
The patent applies different rendering qualities to different regions of the 3D display based on user attention. The interaction processing device determines a first region around the eye gaze region where high definition rendering is applied, while other regions receive low definition rendering. This local quality differentiation maintains visual quality in the important area while significantly reducing overall bandwidth requirements
Solution Approach 2:
The patent makes the high definition rendering region dynamic by continuously tracking the user's eye gaze position. As the user moves their eyes, the first region where high definition rendering is applied shifts accordingly, ensuring that high visual quality is always provided in the area the user is currently viewing while adapting to changing attention patterns
Data Source
AI summary
Provided are a human-machine interaction method, a system, a processing device and a computer readable storage medium, wherein the method includes: controlling a 3D display to output a 3D view to present a virtual target object; receiving a user image taken by a sight tracking camera, and detecting an eye gaze region according to the user image; receiving a hand image taken by a gesture detection camera, and detecting whether a user's hand collides with the virtual target object and/or grabs the virtual target object according to the hand image; controlling playing of the 3D view according to whether human eyes gaze at the virtual target object, the user's hand collides with the virtual target, and grasps the virtual target object.