Eye Gaze Detection Using Inter-Frame Variation to Reduce Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing line-of-sight detection technologies suffer from high power consumption and poor real-time performance, particularly in applications like VR and AR, due to the need for continuous frame-by-frame image analysis of the user's eye region.
Innovation Solution
A method that compares inter-frame variation amounts between current and reference eye image frames to determine a target gaze direction, allowing for the reuse of pre-generated gaze directions and reducing unnecessary computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If continuous frame-by-frame image analysis is performed to achieve real-time gaze detection, then detection accuracy is improved, but power consumption increases and real-time performance deteriorates
Solution Approach 1:
The patent implements periodic action by performing complete gaze detection only at specific intervals (when inter-frame variation exceeds threshold) rather than continuously for every frame. The system alternates between full detection cycles and reuse of previous results, creating a periodic pattern of high-computation operations followed by low-computation operations, thereby reducing average power consumption while maintaining detection accuracy when needed.
Solution Approach 2:
The patent applies preliminary action by pre-calculating and storing gaze directions during periods when computation resources are available, then reusing these pre-generated results during high-power consumption periods. The inter-frame variation calculation is performed in advance to determine when full detection is necessary, allowing the system to prepare detection results beforehand and avoid unnecessary repeated computations.
2Measurement precision
If continuous frame-by-frame image analysis is performed to achieve real-time gaze detection, then detection accuracy is improved, but real-time performance deteriorates
Solution Approach 1:
The system implements periodic action by alternating between full gaze detection operations and reuse of previous detection results based on inter-frame variation thresholds. This creates a rhythmic pattern of computation-intensive operations followed by lightweight operations, improving overall real-time performance by reducing the average computational load while maintaining accurate detection when eye movements are detected.
Solution Approach 2:
The patent applies partial action by performing complete gaze detection only when necessary (when inter-frame variation exceeds threshold) rather than analyzing every single frame in detail. The system performs partial analysis (inter-frame variation check) on all frames but only executes full detection partially - when the variation threshold indicates actual eye movement occurs, thereby improving real-time performance without completely sacrificing detection accuracy.
3Measurement precision
If complete gaze direction calculation is performed for every frame to ensure accuracy, then measurement precision is improved, but computational overhead increases
Solution Approach 1:
The patent applies preliminary action by performing inter-frame variation calculations in advance before committing to full gaze direction computation. This preliminary check allows the system to identify frames that actually contain eye movement information, so that complete gaze direction calculation is only performed when necessary, reducing overall computational overhead while maintaining accuracy for meaningful detections.
Solution Approach 2:
The patent extracts and separates the inter-frame variation calculation step from the complete gaze direction calculation process. By taking out this preliminary analysis step, the system can identify and process only those frames containing actual eye movement, extracting only the necessary computational work and eliminating redundant calculations from frames where the gaze direction has not changed.
Data Source
Figure 1
Figure 2~3
Figure 4~6
AI summary
Embodiments of the invention provides a method, an apparatus of line-of-sight detection, an electronic device and a storage medium, obtaining a current image frame and a first reference image frame, where in the current image frame is an eye image collected in real time, and the first reference image frame is an eye image collected before the current image frame; obtaining an inter-frame variation amount of the current image frame relative to the first reference image frame; and deriving a target gaze direction based on the inter-frame variation amount, where the target gaze direction is a first gaze direction or a second gaze direction, the first gaze direction is a gaze direction pre-generated based on the first reference image frame, and the second gaze direction is a gaze direction generated in real time based on the current image frame. By comparing the inter-frame variation amount between the current image frame and the first reference image frame, the first gaze direction corresponding to the first reference image frame or the second gaze direction corresponding to the current image frame is opportunistically selected as the target gaze direction, the computational overhead during the line-of-sight detection process can be reduced, thereby improving the real-time performance of line-of-sight detection.