Super-Resolution Imaging for OCL Pixel Value Reversal Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image sensors with a 2×2 On Chip Lens (OCL) structure suffer from pixel value reversal due to phase differences, leading to degraded image quality, necessitating improved image processing techniques.
Innovation Solution
An image processing device and method utilizing a super resolution network trained on images from a second image sensor with a different structure to correct and enhance images captured by a first image sensor with phase detection pixels, employing a domain transfer method to address pixel value reversals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a 2×2 On Chip Lens (OCL) structure is used in image sensors, then fast focusing is achieved, but pixel value reversal occurs due to phase differences, deteriorating image quality
Solution Approach 1:
A neural network model is introduced as an intermediary between the defective image data from the 2×2 OCL sensor and the final corrected output. The neural network learns the mapping relationship between images from different sensor types and applies this learning to correct pixel value reversals, effectively mediating the conflict between fast focusing capability and image quality
Solution Approach 2:
The patent changes the parameter of image processing by transitioning from traditional signal processing methods to neural network-based deep learning. This parameter change enables the system to adaptively correct pixel value reversals by learning from training data, resolving the contradiction between maintaining fast focusing speed and improving image quality
2Manufacturing precision
If a neural network model is trained using images from a second image sensor with a different structure, then pixel value reversals are corrected in the first image sensor, but the complexity of the image processing system increases
Solution Approach 1:
Instead of physically modifying the first image sensor or adding complex hardware, the patent creates a virtual copy of the second image sensor's imaging characteristics through neural network training. The neural network learns to replicate the behavior of the well-performing second sensor to correct the first sensor's defects, avoiding direct hardware complexity while achieving image quality improvement
Data Source
AI summary
An image processing device includes a memory configured to store a super resolution network trained to output a corrected image based on an input image; and a processor configured to output a super resolution image based on a first image acquired by a first image sensor and a corrected image output from the super resolution network. The super resolution network is a model trained by using second images acquired through a second image sensor, a type of the second image sensor being different from a type of the first image sensor.