Adversarial Gaze Feature Extraction Network
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current gaze tracking methods using deep learning models often combine gaze-related and irrelevant features, such as lighting, leading to inaccurate and unstable gaze direction predictions.
Innovation Solution
A method involving a feature extraction network optimized through adversarial training, which separates gaze-related features from irrelevant ones by using a combination of image reconstruction and gaze prediction networks to maximize gaze information and minimize image information, ensuring the extracted features contain only relevant gaze data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If deep learning models are used to extract features from face or eye images, then the model can learn complex patterns, but gaze-irrelevant features (such as lighting, personal appearance) become combined with gaze-related features, reducing prediction accuracy
Solution Approach 1:
The patent segments the feature extraction process into two distinct pathways: a standard feature extraction network that captures comprehensive image features, and an adversarial network that specifically targets and removes gaze-irrelevant features. This segmentation allows the system to handle both complex pattern recognition and feature purification separately, resolving the contradiction between adaptability and precision.
Solution Approach 2:
The adversarial network acts as an intermediary component between the feature extraction network and the gaze prediction network. It receives features from the extraction network, identifies and removes gaze-irrelevant information through adversarial training, and passes purified gaze-related features to the prediction network. This intermediary mechanism enables the system to maintain comprehensive feature extraction while eliminating harmful features.
2Loss of information
If comprehensive image features are extracted to ensure all possible gaze information is captured, then more gaze information is available, but irrelevant features (lighting, appearance) are also included, reducing stability
Solution Approach 1:
The patent applies the extraction principle by using the adversarial network to specifically extract and remove gaze-irrelevant features from the comprehensive feature set. The adversarial network is trained to identify features related to lighting, personal appearance, and other irrelevant attributes, and explicitly remove them while preserving gaze-related information. This selective extraction process ensures that only stable, gaze-relevant features are passed to the prediction network.
Solution Approach 2:
The adversarial training process implements a feedback mechanism where the adversarial network receives feedback from the feature extraction network about the presence of irrelevant features, and adjusts its removal strategy accordingly. The system continuously refines which features to remove based on the adversarial objective, creating a feedback loop that progressively improves feature purification while maintaining gaze information stability.
3Reliability
If adversarial training is used to remove gaze-irrelevant features, then feature purity is improved, but the training process becomes more complex
Solution Approach 1:
The patent merges the adversarial training objective with the standard feature extraction process by integrating the adversarial network into the existing deep learning architecture. Rather than creating a completely separate system, the adversarial component is combined with the feature extraction network, allowing them to work together in a unified training framework. This merging reduces overall system complexity while maintaining the benefits of adversarial training.
Data Source
AI summary
The embodiments of the present disclosure disclose a method for determining the direction of gaze. A specific implementation of the method includes: obtaining a face or eye image of a target subject, and establishing a feature extraction network; using an adversarial training method to optimize the feature extraction network, and implicitly removing the gaze-irrelevant features extracted by the feature extraction network, so that the feature extraction network extracts gaze-related features from the face or eye image to obtain the gaze-related features; determining the target gaze direction based on the gaze-related features. This implementation can separate the gaze-irrelevant features contained in the image features from the gaze-related features, so that the image features contain the gaze-related features, and that the accuracy and stability of the determined direction of gaze are further improved.