Gaze Detection Model Training Using Head and Body Direction Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional gaze detection methods are limited in applicability and accuracy, particularly when the pupil is not captured in the image, and do not effectively utilize body and head direction information for improved gaze detection.
Innovation Solution
A deep learning-based gaze detection model is trained using body and head direction information, employing convolutional and fully connected layers to generate feature maps and predict gaze directions, with loss weights and ground truths for model training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional gaze detection methods using only facial images are employed, then the system is simple to implement, but the applicability is extremely limited and accuracy deteriorates when the pupil is not captured
Solution Approach 1:
The patent segments the feature extraction process into two independent parallel streams: one for body direction information and one for head direction information. Each stream processes its respective input data through separate convolutional and fully connected layers, allowing the system to independently analyze body and head orientations without requiring one another, thereby improving applicability while maintaining accuracy through multi-source fusion.
Solution Approach 2:
The patent creates a universal gaze detection system that can handle multiple input types (body images, head images, or both combined) and multiple detection scenarios (pupil visible, pupil not visible, various viewing angles). The model architecture is designed to be multi-functional, capable of processing different image configurations and producing reliable gaze predictions across diverse conditions, thus resolving the contradiction between broad applicability and consistent accuracy.
2Adaptability or versatility
If pupil-based gaze detection from facial images is used, then the method works under specific conditions, but it becomes inapplicable when the pupil is not captured in the image
Solution Approach 1:
The patent introduces body direction information as an intermediary element that mediates between the available visual data (body/head images) and the target measurement (gaze direction). When pupil-based detection fails or is unavailable, the body direction stream provides alternative directional cues that can compensate for the missing pupil information, ensuring reliable gaze detection across various environments including cases where the pupil is not captured.
Solution Approach 2:
The patent performs preliminary feature extraction and direction estimation from body and head images before attempting final gaze detection. By pre-processing the input images to extract body direction and head direction features in advance, the system prepares alternative detection pathways that can be activated when pupil-based detection is not feasible, thereby maintaining reliability across different environmental conditions.
3Measurement precision
If a deep learning model with multiple convolutional and fully connected layers is trained using body and head direction information, then the gaze detection accuracy is improved, but the device complexity increases
Solution Approach 1:
The patent divides the complex deep learning model into segmented functional modules: body image input module, head image input module, separate convolutional layers for each modality, separate fully connected layers for direction prediction, and a fusion mechanism. This segmentation allows each module to be optimized independently and trained separately, reducing the overall training complexity while maintaining high detection accuracy through specialized feature extraction in each segment.
Solution Approach 2:
The patent implements a flexible training approach where the system can perform partial training using only body images or only head images if desired, or excessive training by utilizing both modalities together for enhanced accuracy. The model architecture supports selective activation of different components, allowing users to balance between training complexity and detection accuracy based on available data and computational resources, thus resolving the contradiction between model complexity and performance.
Data Source
AI summary
Method for training a deep learning-based gaze detection model includes steps of: (a) generating body direction loss by using predicted body direction information and labeled body direction information included in first ground truth corresponding to the first training image, to thereby train a body FC layer and a body convolutional layer; and (b) inputting a first integrated feature map into a head FC layer, to thereby instruct the head FC layer to perform an FC operation on the first integrated feature map and thus output first predicted head direction information which is acquired by predicting a direction in which a front of a head of a second person is directed, and generating head direction loss by using the first predicted head direction information and labeled head direction information included in second ground truth corresponding to the second training image, to thereby train the head FC layer and a head convolutional layer.