Eye Gaze Correction in Video Conference Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze correction techniques in video conferencing fail to dynamically adjust eye gaze to simulate natural in-person interactions, leading to unnatural sustained eye contact and neglecting expected eye movements like blinking and brief glances, which can result in discomfort for participants.
Innovation Solution
The system evaluates a user's eye gaze during video conferences, detects the direction of gaze, and generates gaze-corrected frames to simulate eye contact by adjusting the eyes' orientation, while also providing alerts to ensure the user maintains appropriate eye contact durations based on social feedback models and neural network analyses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If gaze correction techniques are applied to simulate eye contact, then eye contact simulation is improved, but naturalness of interaction deteriorates due to sustained eye contact
Solution Approach 1:
The system dynamically adjusts eye gaze correction based on detected eye movement patterns, transitioning between corrected and uncorrected states to simulate natural interaction. The gaze correction is applied selectively rather than continuously, allowing the system to adapt to natural blinking and gaze shifts while maintaining eye contact simulation where appropriate.
Solution Approach 2:
The system incorporates feedback mechanisms that detect actual eye movement patterns and adjust gaze correction accordingly. By monitoring eye gaze direction, blinking patterns, and interaction context, the system receives feedback about natural interaction behavior and modifies its gaze correction output to maintain naturalness while preserving eye contact simulation effectiveness.
2Reliability
If gaze correction is applied continuously, then eye contact simulation is improved, but discomfort for participants increases
Solution Approach 1:
The system applies gaze correction in periodic intervals rather than continuously. By detecting natural eye movement patterns such as blinking and gaze shifts, the system applies correction only during appropriate periods when eye contact is socially appropriate, and suspends correction during periods when natural eye movement indicates a break in eye contact is needed, thereby preventing participant discomfort.
Solution Approach 2:
The system changes the parameter of gaze correction application based on detected interaction context and eye movement patterns. When natural eye movement patterns indicate sustained eye contact is appropriate, the correction parameter is activated; when patterns indicate natural breaks or shifts, the correction parameter is deactivated or reduced, thus preventing discomfort while maintaining simulation effectiveness.
3Reliability
If gaze correction is applied to all video frames, then eye contact simulation is improved, but processing complexity increases
Solution Approach 1:
The system extracts and processes only the portions of video frames that contain eye regions rather than processing all frames uniformly. By identifying and isolating the eye area within each frame and applying gaze correction only to this extracted region, the system significantly reduces processing complexity while maintaining effective eye contact simulation in the relevant visual information.
Solution Approach 2:
The system applies different processing quality to different regions of the video frame. Instead of uniform processing, the gaze correction is applied locally only to the eye region with high precision, while other regions undergo minimal or no processing. This local quality approach maintains effective eye contact simulation while reducing overall processing complexity and computational load.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, apparatus, articles of manufacture, and methods for eye gaze correction in camera image streams are disclosed. An example apparatus disclosed herein includes timing circuitry (306) to calculate a duration of time for which an eye gaze (128) of a first user (104) of a video conference is directed toward a second user of the video conference, the second user presented as an image (116A) via a video conference interface (117), the video conference interface presented by a display (114) of an electronic device (112), and social feedback control circuitry (310) to select an eye gaze time threshold based on content associated with the video conference interface, and in response to the duration satisfying the eye gaze time threshold, cause an alert to be presented on the display with the video conference interface.