Dynamic ISP Camera Pipeline for AR Eyewear Power Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Portable eyewear devices, such as smart glasses, face challenges in managing power consumption due to the processing of various features, leading to increased power usage and thermal issues, particularly in camera operations for augmented reality and computer vision applications.
Innovation Solution
The implementation of a dynamic image signal processor (ISP) in a camera pipeline that operates in both high-resolution augmented reality (AR) and low-resolution computer vision (CV) modes, with power-saving features like image downscaling in cameras, disabling auto-exposure and auto-white balance for CV modes, and dynamic mono/stereo camera pipeline management to reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the camera operates in high-resolution AR mode, then image quality is improved, but power consumption increases
Solution Approach 1:
The camera pipeline dynamically switches between AR mode and CV mode based on operational requirements. The system adjusts resolution, enables/disables auto-exposure and auto-white balance, and transitions between mono and stereo configurations to optimize power consumption while maintaining image quality when needed.
Solution Approach 2:
The system changes operational parameters including resolution (high for AR, low for CV), processing features (full auto-exposure/AWB for AR, disabled for CV), and camera configuration (stereo for AR, mono for CV) to resolve the contradiction between image quality and power consumption.
2Use of energy by moving object
If the camera operates in low-resolution CV mode, then power consumption is reduced, but image quality deteriorates
Solution Approach 1:
The system dynamically adjusts operational modes based on task requirements. For computer vision applications where high image quality is not critical, the system switches to low-resolution CV mode with reduced processing features, thereby consuming less power while accepting lower image quality.
Solution Approach 2:
The system changes parameters to low resolution, disables auto-exposure and auto-white balance processing, and uses mono camera configuration for CV mode to reduce power consumption, accepting the trade-off of reduced image quality for the benefit of extended runtime.
3Measurement precision
If high-resolution images are processed continuously, then AR experience quality is improved, but thermal constraints are exceeded
Solution Approach 1:
The system uses periodic action by switching between high-power AR mode and low-power CV mode. High-resolution processing is performed only when AR experience is required, while CV algorithms run in lower-power mode otherwise, preventing continuous thermal buildup while maintaining AR quality when needed.
Solution Approach 2:
The camera pipeline dynamically adjusts its operational state based on thermal conditions and application requirements, transitioning between full AR processing and reduced CV processing to manage thermal constraints while preserving AR experience quality.
Data Source
AI summary
Eyewear having an image signal processor (ISP) dynamically operable in a camera pipeline for augmented reality (AR) and computer vision (CV) systems. Multi-purpose cameras are used for simultaneous image capture and CV on wearable AR devices. The cameras are coupled to a frame and configured to generate images, wherein the cameras and the ISP operate in a first AR mode and capture images having a first resolution suitable for use in AR, and are configured to operate in a second CV mode to provide the images having a second resolution suitable for use in CV. The first resolution in the AR mode is higher than the second resolution in the CV mode, and the cameras and the ISP consume less power in the second CV mode than the first AR mode. The cameras and the ISP save significant system power by operating in the low power mode CV mode.