Neural Gaze Prediction Using Saliency Maps for Video Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Determining user actions or focus in various applications is challenging, impacting the accuracy of assistance and automation, particularly in predicting gaze direction and attention in video content.
Innovation Solution
Training neural networks using gaze data from viewers to infer gaze locations and attention regions, utilizing separate models for different types of content and incorporating saliency maps to modify video rendering and compression based on viewer attention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gaze data is collected and processed to determine user attention and focus, then the accuracy of user behavior prediction is improved, but the computational complexity and data processing requirements increase
Solution Approach 1:
The patent divides the gaze determination task into multiple separate neural network models, each specialized for different types of content (e.g., video content model, image content model). This segmentation allows each model to be optimized for specific content types, improving accuracy while managing complexity through modular architecture.
Solution Approach 2:
The patent implements content-type-specific neural network models that are tailored to process different kinds of visual content differently. Each model has specialized parameters and architectures optimized for its designated content type, providing locally optimized processing rather than a generic one-size-fits-all approach.
2Measurement precision
If multiple separate neural network models are used for different content types, then the accuracy of gaze prediction for specific content is improved, but the number of models and system complexity increases
Solution Approach 1:
The patent creates a universal gaze determination system that can handle multiple content types through separate specialized models. Each model is designed to work with specific content types (video, images, etc.), but the overall system provides versatile coverage across different media formats, achieving multi-functionality through modular specialization.
3Manufacturing precision
If saliency maps are generated and applied to modify video rendering and compression, then the viewing quality in attention regions is improved, but the processing time and computational resources increase
Solution Approach 1:
The patent applies saliency maps to selectively modify video rendering and compression parameters in different spatial regions. High-quality rendering is applied specifically to attention regions identified by the saliency maps, while non-attention regions receive standard processing. This local quality approach maintains overall video quality in important areas while reducing total processing time and resources.
Data Source
AI summary
Apparatuses, systems, and techniques are presented to predict gaze of an observer. In at least one embodiment, a network is trained to predict a gaze of one or more users based, at least in part, on one or more gazes corresponding to objects not always visible to the one or more users.