Dynamic Image Segmentation for Power-Efficient AR Vision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video and augmented reality systems face challenges in efficiently capturing and processing image data, particularly in managing power consumption and optimizing image quality across varying motion and depth detection, which affects the performance and battery life of head-wearable devices.
Innovation Solution
The system segments the field of view into regions with different frame rates and image capture qualities based on motion detection, depth estimation, and audio sources, using both local and remote processing circuitry to adjust settings dynamically and reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the camera sensor captures full frames at full resolution and maximum frame rate, then image quality is improved, but power consumption increases
Solution Approach 1:
The field of view is divided into multiple regions of interest (ROIs) based on motion detection, depth estimation, and audio sources. The camera sensor captures image data only for these specific regions at high frame rates and resolutions, while other regions are captured at lower frame rates or skipped entirely. This segmentation approach maintains image quality for critical areas while significantly reducing overall power consumption.
Solution Approach 2:
Different regions of the field of view are assigned different capture qualities based on their importance. Regions containing motion, foreground objects, or audio sources receive high frame rates and full resolution, while background regions receive lower frame rates or are skipped. This local quality differentiation optimizes the balance between image quality and power consumption.
2Measurement precision
If the system processes all regions of the field of view uniformly, then image quality is maintained, but processing efficiency decreases
Solution Approach 1:
The field of view is segmented into multiple regions of interest based on motion detection, depth estimation, and audio source localization. The processing system applies different frame rates and processing intensities to different regions, rather than uniformly processing all regions. This segmentation enables the system to maintain image quality for critical regions while improving overall processing efficiency by reducing the total amount of data that requires high-intensity processing.
Solution Approach 2:
Different processing qualities are applied to different regions based on their importance. High-priority regions (containing motion, foreground objects, or audio sources) receive full processing resources, while low-priority background regions receive reduced processing. This local quality approach maintains image quality where needed while improving processing efficiency overall.
3Speed
If the camera operates at maximum frame rate for the entire field of view, then motion capture accuracy is improved, but battery life decreases
Solution Approach 1:
The field of view is divided into multiple regions, and the camera sensor captures image data at different frame rates for different regions. Regions containing motion or audio sources are captured at high frame rates to maintain motion capture accuracy, while static background regions are captured at lower frame rates or skipped entirely. This selective frame rate approach preserves battery life while maintaining motion capture accuracy where needed.
Solution Approach 2:
The frame rate assigned to different regions is dynamically adjusted based on detected motion, depth, and audio sources. Regions that become active (containing motion or audio) automatically receive higher frame rates, while inactive regions receive lower frame rates. This dynamic adjustment optimizes the balance between motion capture accuracy and battery life throughout the video sequence.
Data Source
AI summary
A method for obtaining multiple frames of image data includes obtaining a first frame of image data of an entirety of a field of view by a camera sensor. The method also includes defining one or more regions of the field of view using the first frame of image data. The method includes determining at least one of a frame rate and an image capture resolution for each of the one or more regions. The method includes operating the camera sensor to obtain partial frames of image data by obtaining image data of the one or more regions of the field of view according to at least one of the frame rate and the image capture resolution, and re-constructing one or more full frames from partial frames of image data.