Multi-Frame Demosaicing for Digital Zoom Aliasing Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile electronic devices suffer from aliasing artifacts in image capture due to digital zoom and demosaicing operations, particularly in scenes with edges, degrading image quality.
Innovation Solution
A combined intraband and interband multi-frame demosaicing method that includes intraband joint demosaic-warp operations to reconstruct image data and blend RGB frames, followed by interband demosaic operations to equalize high-frequency content across color channels, reducing aliasing and enhancing image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If demosaicing operations are performed to reconstruct image data from color filter arrays, then complete color information is achieved, but aliasing artifacts are introduced degrading image quality
Solution Approach 1:
The patent segments the demosaicing process into two distinct stages: intraband demosaicing performed on individual frames to reconstruct color information, and interband demosaicing performed after frame blending to equalize high-frequency content. This segmentation allows each stage to address specific requirements without introducing the full range of artifacts that would result from a single comprehensive demosaicing operation.
Solution Approach 2:
The patent performs preliminary intraband demosaicing operations on individual frames before blending, which reconstructs the color information needed for subsequent blending operations. This preliminary action ensures that the blending process has complete color data to work with, while the final interband demosaicing equalizes high-frequency content after blending to remove aliasing artifacts.
2Length of moving object
If digital zoom is used to capture images, then close-up scenes are achievable, but aliasing artifacts are created during image generation
Solution Approach 1:
The patent transitions from single-frame processing to multi-frame processing, adding the temporal dimension to the image capture and processing pipeline. By capturing and processing multiple frames at different zoom levels or positions, the system can reconstruct high-frequency detail information that would be lost in a single digital zoom operation, thereby reducing aliasing artifacts while maintaining zoom capability.
3Loss of information
If intraband demosaicing is performed to reconstruct color channels, then complete RGB data is produced, but high-frequency content is not equalized across channels
Solution Approach 1:
The patent extracts and processes different frequency components at different stages. The intraband demosaicing extracts and reconstructs color information for each channel independently, while the subsequent interband demosaicing specifically targets and equalizes the high-frequency content across channels. This extraction approach allows each processing stage to focus on its specific requirement without compromising the other.
4Object-affected harmful factors
If multiple frames are blended to reduce aliasing, then aliasing artifacts are removed, but processing complexity increases
Solution Approach 1:
The patent segments the processing pipeline into distinct stages with clear responsibilities: frame capture, intraband demosaicing, frame blending, and interband demosaicing. This segmentation allows each stage to be optimized independently and makes the overall complex process more manageable and implementable in real-time systems.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A method includes obtaining multiple input image frames and determining how to warp at least one of the input image frames. The method also includes performing an intraband demosaic-warp operation to reconstruct image data in different color channels of the input image frames and warp the at least one input image frame to produce RGB input image frames. The method further includes blending the RGB input image frames to produce a blended RGB image frame, performing an interband denoising operation to produce a denoised RGB image frame, and performing an interband sharpening operation to produce a sharpened RGB image frame. In addition, the method includes performing an interband demosaic operation to substantially equalize high-frequency content in different color channels of the sharpened RGB image frame to produce an equalized sharpened RGB image frame and generating a final image of the scene based on the equalized sharpened RGB image frame.