Defect Pixel Detection in Microlens Array Image Sensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In image pickup apparatuses with microlens arrays, real-time defect pixel detection is challenging due to the rearrangement of pixel signals during refocus processing, making it difficult to determine whether a pixel is defective based on peripheral pixel signals, especially when the image data is not suitable for direct display.
Innovation Solution
An image processing apparatus that includes a refocus processing unit and a defect pixel detection unit, allowing for the detection of defect pixels in the picked-up image and performing refocus processing only after defect pixel detection, using standard deviation calculations to identify defective pixels within unit pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real-time defect pixel detection is performed on picked-up image data from image pickup elements with microlens arrays, then defect pixels can be identified, but the detection becomes unreliable because pixel signals are rearranged during refocus processing and the image data is not suitable for direct display
Solution Approach 1:
The patent performs defect pixel detection before refocus processing is applied to the image data. By detecting defects in the original picked-up image data while it is still in a suitable state for direct display and observation, the system avoids the problem of signal rearrangement that occurs during refocus processing. This preliminary detection ensures reliable identification of defect pixels based on actual pixel signal levels rather than rearranged data.
2Ease of operation
If refocus processing is performed first on the picked-up image, then the image can be reconstructed for display, but the pixel signal arrangement no longer corresponds to the original positional relationships making defect detection difficult
Solution Approach 1:
The system performs defect pixel detection before refocus processing, establishing the detection task in advance when the image data is still in its original, display-suitable form. This ensures that defect pixels are identified based on their actual signal characteristics before any rearrangement occurs during refocus processing.
Solution Approach 2:
Instead of performing refocus processing first and then attempting defect detection on the rearranged data, the patent inverts the sequence by detecting defects first and then applying refocus processing. This inversion of the conventional processing order resolves the contradiction by maintaining the original pixel signal relationships needed for reliable detection.
3Productivity
If defect pixel detection is performed on rearranged pixel data after refocus processing, then processing can be done on reconstructed images, but the level difference between detection subject pixel and peripheral pixel becomes insufficient for accurate discrimination
Solution Approach 1:
The patent performs defect pixel detection before refocus processing, when the pixel data is still in its original state with adequate signal levels for discrimination. By completing the detection task preliminarily, the system avoids working with rearranged data where level differences are insufficient for accurate defect identification.
Data Source
AI summary
An image processing apparatus for processing a picked-up image which is output from an image pickup element in which a microlens array for obtaining the picked-up image including ray directional information of an object image formed by a photographing optical system has been arranged has: a refocus processing unit for executing a refocus processing of the obtained picked-up image on the basis of the ray directional information and generating a reconstruction image; and a defect pixel detection unit for detecting a defect pixel from the obtained picked-up image. When the picked-up image is obtained, the detection of the defect pixel of the obtained picked-up image is performed. When a refocus processing is instructed, the refocus processing is executed to the picked-up image in which the detection of the defect pixel is performed.