Gaze-Based Video Block Update Rate Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video monitoring systems consume significant network resources due to the distribution of video streams, even when not all streams are being viewed by the operator, leading to inefficient use of processing and network resources.
Innovation Solution
The method involves using gaze sensing to determine the operator's focus and reduce the update rate of video stream blocks not in the operator's primary view, by designating skip block insertion points based on the gaze point, which reduces the bitrate and computational load on both the encoder and decoder.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video streams are distributed to all operators regardless of viewing status, then complete video monitoring coverage is maintained, but network resources are significantly consumed
Solution Approach 1:
The patent applies local quality by differentiating video block update rates based on operator gaze position. Blocks corresponding to the foveal vision area (where the operator is looking) are updated at full rate, while blocks in peripheral areas are updated at reduced rates or skipped entirely. This selective update approach maintains monitoring reliability for viewed areas while reducing network resource consumption for non-viewed areas.
2Manufacturing precision
If full update rate video streams are transmitted, then video quality and detail are maintained, but computational load on encoder and decoder increases
Solution Approach 1:
The patent segments the video stream into multiple blocks and applies different update rates to different blocks based on gaze position. The video frame is divided into blocks, and skip block insertion points are designated in peripheral areas while maintaining full updates in the foveal vision area. This segmentation allows the system to maintain video quality where needed while reducing computational load in less critical areas.
3Measurement precision
If video blocks are updated at full rate for all areas, then motion detection accuracy is maintained, but power consumption increases
Solution Approach 1:
The patent implements dynamic update rates for video blocks based on real-time gaze detection. The system continuously monitors operator eye position and adjusts the update rate of video blocks accordingly. When the operator looks at a particular area, blocks in that area are updated at full rate for accurate motion detection, while blocks in other areas are updated at reduced rates, dynamically adapting to operator attention and reducing power consumption.
Data Source
AI summary
A method decodes video data based on gaze sensing. The method may decode an encoded video stream received from an encoder associated with a camera, and present the decoded video stream on a display of a device. The method may further detect a gaze point of an operator viewing the display, designate locations associated with the decoded video stream, based upon the gaze point, as skip block insertion points. The method may send the locations to the encoder, wherein the encoder reduces an update rate of inter-frame coded blocks corresponding to the skip block insertion points when encoding video data produced by the camera. An apparatus can implement the method to decode video data based on gaze sensing.