Dual Image Capture Flows for High-Res Processing and Live Preview
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image capture devices struggle with efficient processing of high-resolution raw images and simultaneous display of image previews, particularly in spherical image capture devices, leading to inefficiencies and suboptimal user experience.
Innovation Solution
Implementing a method and device architecture that includes separate DDR memory buffers for high and low resolution YUV data processing, allowing for delayed and live processing to manage image data effectively, enabling high-resolution image processing while displaying previews.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-resolution raw images are processed in real-time, then image quality is improved, but processing time and computational load increase
Solution Approach 1:
The processing system is segmented into two independent pipelines: a first processing path that performs comprehensive high-resolution processing (including denoising, sharpening, and color correction) for final image output, and a second processing path that performs simplified real-time processing for preview display. This segmentation allows each path to be optimized for its specific purpose without compromising the other.
Solution Approach 2:
The second processing path applies partial processing actions - only essential operations are performed on the preview stream (basic color correction and scaling) while omitting computationally intensive operations like advanced denoising and sharpening. This partial action enables real-time preview generation with acceptable quality, reserving full processing for the first path.
2Manufacturing precision
If high-resolution video data is stored in memory buffer, then image quality is improved, but memory bandwidth and storage requirements increase
Solution Approach 1:
The memory buffer system is segmented into two separate buffers: a first DDR memory buffer dedicated to storing high-resolution YUV video data for the first processing path, and a second DDR memory buffer for storing low-resolution YUV video data for the second processing path. This segmentation prevents memory bandwidth contention and allows each buffer to be optimized for its specific resolution requirements.
Solution Approach 2:
The system creates a copy of the raw video data stream, directing one copy to the first processing path for high-quality processing and another copy to the second processing path for preview generation. This copying approach allows simultaneous processing at different resolutions without requiring the full high-resolution data to be processed for every output stream.
3Area of moving object
If spherical image capture is implemented, then field of view is improved, but image processing complexity increases
Solution Approach 1:
The spherical image processing is segmented into distinct operational modes: a first processing mode that handles full spherical projection transformation and rendering for the complete field of view, and a second processing mode that extracts and processes only a planar portion of the spherical data for standard display devices. This segmentation allows the system to provide comprehensive spherical capture capability while offering optimized processing paths for different use cases.
Solution Approach 2:
The second processing path extracts only the necessary planar portion from the spherical image data, removing unnecessary spherical transformation computations. This extraction approach reduces processing complexity for applications that don't require the full spherical field of view, such as standard smartphone displays, while preserving the complete spherical data in the first processing path for applications that do require it.
Data Source
AI summary
Image processing using various video and still flows is described. The resolution and bit depth at each stage of the image processing are described. In some examples, image scalers are used to resize image resolution. In some examples, a warp engine is used to distort per frame images to apply image stabilization, zoom, or a user digital lens. An image processing pipeline includes a double data rate (DDR) memory buffer that supports lossy compression with a constant 50% compression. In some examples, the image processing pipeline includes a DDR memory buffer that is uncompressed.