Video Encoding Using Historical Gaze Heat Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Monitoring systems with multiple cameras face high network bandwidth consumption and resource utilization due to the need to transmit and process video streams, with existing methods failing to efficiently manage resource allocation based on operator interest in different areas of the video stream.
Innovation Solution
A method using eye tracking sensors to generate historical gaze heat maps, which identify low and high interest areas, allowing cameras to adjust bit rates accordingly, reducing resource usage in low interest areas and increasing it in high interest areas, thereby conserving network resources and reducing processor and memory loads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the bit rate of the video stream is maintained at a high level across the entire stream, then the quality of the video stream is preserved, but network bandwidth consumption and resource utilization increase significantly
Solution Approach 1:
The patent applies local quality by adjusting the bit rate differently across different spatial regions of the video stream. Specifically, it identifies operator gaze areas and allocates higher bit rates to these regions while using lower bit rates for non-gaze regions, thereby maintaining perceived video quality while reducing overall network bandwidth consumption.
Solution Approach 2:
The patent implements dynamic bit rate adjustment based on real-time operator gaze tracking. The system continuously monitors where the operator is looking and dynamically reallocates bandwidth resources to match the operator's attention patterns, transitioning from static uniform bit rate allocation to dynamic region-specific allocation.
2Loss of energy
If the bit rate is decreased in low interest areas, then network resources are conserved, but the quality of those areas deteriorates
Solution Approach 1:
The patent applies local quality by making the video quality adaptive to the operator's attention. In low interest areas (where the operator is not looking), the system reduces video quality through lower bit rates, while maintaining high quality in gaze areas. This creates a non-uniform quality distribution that matches operator attention patterns.
3Productivity
If eye tracking sensors and historical gaze heat map generation are implemented, then resource allocation can be optimized based on operator interest, but device complexity and processing requirements increase
Solution Approach 1:
The patent introduces an intermediary component (the eye tracking sensor and gaze heat map generation system) that bridges the operator's attention state and the video encoding parameters. This intermediary processes gaze data and translates it into bit rate allocation decisions, enabling intelligent resource optimization without requiring complex manual configuration.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and a video management system is disclosed. The method may include receiving a video stream from a camera and displaying the video stream on a display. The method may include obtaining, via an eye tracking sensor, gaze information for an operator watching the display. The method may include generating a historical gaze heat map for the video stream for a time period based on the obtained gaze information and determining a low interest area for the video stream based on the generated historical gaze heat map. The method may include instructing the camera to decrease a bit rate of the video stream in the low interest area.