Stereo Camera Complementary Subsampling for Fast Wide-FOV Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image capturing and processing technology is inefficient in generating high-quality images with a wide field of view, high resolution, and high framerate, leading to poor user experience in XR devices due to high computational burden and resource requirements.
Innovation Solution
An imaging system and method utilizing complementary subsampling in stereo cameras, where each camera has a different subsampling pattern, followed by interpolation and demosaicking to generate high-quality images with a wide field of view and high framerate, reducing computational burden and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If full pixel readout is used to generate high-resolution images, then image quality is improved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent applies partial action by implementing selective pixel subsampling where only a subset of pixels (e.g., every second or third pixel) is fully read out and processed, while other pixels are subsampled or skipped. This partial processing approach maintains acceptable image quality for the central field of view while dramatically reducing processing time and computational load, directly resolving the contradiction between full pixel readout quality and processing efficiency.
Solution Approach 2:
The patent segments the field of view into different regions (central/high-priority and peripheral/low-priority) and applies different processing strategies to each segment. The central region receives full or near-full pixel readout for high quality, while peripheral regions use aggressive subsampling. This segmentation allows the system to optimize processing time while maintaining quality where it matters most.
2Manufacturing precision
If full pixel readout is used to achieve wide field of view with high resolution, then visual quality is improved, but the total number of pixels that can be processed at a given frame rate is limited
Solution Approach 1:
The patent implements partial action by processing only a subset of pixels at full resolution while subsampling others, enabling the system to handle a much larger total number of pixels across the wide field of view. This allows maintaining high frame rates (e.g., 90 FPS) while achieving wide field of view (130 degrees × 105 degrees) with acceptable visual quality through selective full processing of only necessary pixels.
Solution Approach 2:
The patent applies local quality by providing high processing quality (full pixel readout) only in regions where it is most needed (central field of view, foveal region) while using lower quality processing (subsampling) in peripheral regions. This localized quality approach enables the system to achieve wide field of view at high frame rates while maintaining high visual quality where the human eye is most sensitive.
3Manufacturing precision
If high-resolution images are generated for wide field of view, then visual quality is improved, but computational power requirements and processing resources increase
Solution Approach 1:
The patent applies partial action by performing computationally intensive full pixel readout and processing only on a subset of pixels that contribute most to perceived image quality, while using computationally lighter subsampling methods for remaining pixels. This dramatically reduces the computational power and processing resources required while maintaining high visual quality in critical regions.
Solution Approach 2:
The patent implements local quality by concentrating computational power and processing resources on regions of the image where high visual quality is most important (central field of view) while using minimal processing resources for peripheral regions. This localized resource allocation enables high visual quality where needed while keeping overall computational power requirements manageable for XR devices.
4Loss of time
If subsampling is applied to reduce processing burden, then processing time is reduced, but image quality deteriorates
Solution Approach 1:
The patent applies local quality by using full or near-full pixel readout (maintaining high image quality) in the central field of view where the human eye has highest acuity, while applying subsampling (reducing processing time) in peripheral regions where visual acuity is lower. This spatially varying processing strategy reduces overall processing time while maintaining high image quality in the most visually important regions.
Solution Approach 2:
The patent implements partial action by applying full processing (no subsampling) only to the necessary subset of pixels in the central region, while applying subsampling to the remaining pixels in peripheral regions. This partial application of full processing maintains image quality where it matters most while reducing processing time through subsampling elsewhere, resolving the contradiction between processing time and image quality.
Data Source
AI summary
First image data including first subsampled image data of at least a first part of a first field of view a first image sensor is obtained. Second image data including second subsampled image data of at least a second part of a second field of view of a second image sensor is obtained. The first part and the second part have at least a part of an overlapping field of view. The first image data and the second image data are processed to generate a first image and a second image, respectively. During processing, interpolation and demosaicking on the first subsampled image data is performed by utilising the second subsampled image data, while interpolation and demosaicking on the second subsampled image data is performed by utilising the first subsampled image data.