Video Conferencing With Foreground-First Processing for Power Savings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video conferencing on battery-powered mobile devices consumes significant power due to continuous processing of both foreground and background elements, which can be optimized to extend battery life without compromising image quality or latency.
Innovation Solution
A method to classify video frames as having static or changing backgrounds, processing the foreground at full frame rate and the background at reduced frame rate by using semantic segmentation and motion estimation to determine a foreground bounding box, and selectively processing the background only when motion is detected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If continuous processing of both foreground and background elements is performed, then image quality is maintained, but power consumption increases significantly
Solution Approach 1:
The patent segments the video frame into foreground and background regions, applying different processing strategies to each. The foreground region (containing moving objects) is processed at full frame rate to maintain image quality, while the background region is processed at reduced frame rate or skipped when static, thereby reducing overall power consumption without significantly compromising image quality
Solution Approach 2:
The patent applies local quality by differentiating processing intensity across different spatial regions of the video frame. High-quality processing is applied locally to the foreground region where motion occurs, while lower-quality or skipped processing is applied to the static background region, optimizing the trade-off between image quality and power consumption
2Productivity
If full frame rate processing is applied to entire video frames, then image quality is maintained, but processing time and computational resources increase
Solution Approach 1:
The patent divides the video frame into foreground and background segments, allowing differential processing. By identifying and isolating the foreground region containing motion, the system processes only necessary portions at full quality, improving processing efficiency while maintaining image quality where it matters most
Solution Approach 2:
The patent applies partial action by processing only the necessary portions of the video frame at full quality. Instead of uniformly processing entire frames, the system applies full processing only to foreground regions with motion, while using reduced processing for static background regions, thereby improving productivity without excessive sacrifice of image quality
3Use of energy by moving object
If background processing is performed at full frame rate, then background quality is maintained, but power savings are reduced
Solution Approach 1:
The patent introduces dynamic processing of the background by adjusting the processing frame rate based on detected motion. When no motion is detected in the background, processing is reduced or skipped to maximize power savings. When motion is detected, processing is restored to maintain background quality, creating a dynamic adaptation strategy
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Unique characteristics of a video in video conferencing can be exploited for power savings by reducing the rate of processing of a background. The background, if sufficiently static enough, the signal processing of the background can be skipped to save power. Skipping the background means the signal processing processes a cropped image having the foreground only. As a result, the foreground would be processed by the signal processing at a full frame rate, while the background is processed at a reduced frame rate. Optionally, the processed cropped image can be blended with a stored image. Optionally, the processed cropped image can be output without blending.