Nona-pixel color filter array for plenoptic sensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional plenoptic cameras face challenges in demosaicing images, particularly in highly textured areas, due to the limitations of Bayer color filters, which result in inconsistent color balance and spatial resolution in refocused images.
Innovation Solution
A color filter array with a 6×6 pattern of filter pixels, arranged to ensure each sub-aperture image has an extended Bayer pattern, allowing for balanced contributions of red, green, and blue pixels, and the use of micro-lenses covering 3×3 arrays of sensor pixels to capture plenoptic images, facilitating improved demosaicing and refocusing processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a conventional Bayer color filter is used in a plenoptic camera, then the camera can capture light field data, but the demosaicing process becomes complex and produces inconsistent color balance and spatial resolution in refocused images
Solution Approach 1:
The sensor array is divided into multiple sub-arrays, each with a specific color filter pattern (e.g., 2x2 blocks of red, green, blue, and yellow filters). This segmentation allows each sub-array to capture color information independently, simplifying the demosaicing process while maintaining consistent color balance across refocused images at different depths.
Solution Approach 2:
Different regions of the sensor array use different color filter patterns optimized for their specific function. For example, some regions use patterns optimized for color accuracy while others use patterns optimized for spatial resolution. This local optimization resolves the contradiction by allowing each region to excel at its specific task without compromising overall system performance.
2Measurement precision
If a conventional Bayer color filter is used in a plenoptic camera, then the camera can capture light field data, but the spatial resolution in refocused images becomes inconsistent
Solution Approach 1:
The sensor is segmented into multiple sub-arrays with different color filter patterns, where each sub-array contributes to different spatial frequency information. This segmentation enables consistent spatial resolution across refocused images by distributing the spatial sampling burden across multiple specialized sub-arrays, reducing demosaicing complexity.
Solution Approach 2:
The patent introduces a fourth dimension (spectral dimension) by incorporating multiple color filters per spatial location (e.g., 2x2 blocks of different colors). This dimensional expansion allows the system to capture both color and spatial information more efficiently, resolving the contradiction between color balance and spatial resolution consistency while simplifying processing.
3Loss of information
If each pixel is covered by a single color filter, then the camera can capture light field data, but obtaining all three color values for each pixel requires complex demosaicing processing
Solution Approach 1:
The sensor array is segmented into sub-arrays where each sub-array contains multiple color filters (e.g., red, green, blue, yellow) arranged in specific patterns. This segmentation ensures that each spatial location has access to multiple color values through the light field data, reducing information loss while keeping demosaicing processing simpler compared to conventional single-filter-per-pixel approaches.
Solution Approach 2:
The patent uses the light field property of capturing multiple views of the same scene to effectively 'copy' color information across different spatial locations. By combining information from multiple micro-lens images, the system reconstructs complete color values for each pixel without requiring complex demosaicing, as the light field data naturally provides redundant color information from different angles.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution enables well-balanced color contributions in refocused images, improving demosaicing efficiency and maintaining consistent color patterns across different focalization distances, thereby enhancing image quality and stability.
Implementation Method 1
each filter pixel having either a first, a second, or a third color... each of the filter pixels overlays a corresponding one of the sensor pixels
Implementation Method 2
an array of micro-lenses, wherein each of the micro-lenses overlays a respective 3×3 quadrant within the 6×6 pattern of filter pixels
Data Source
AI summary
Example embodiments provide a color filter pattern for a plenoptic sensor. In some embodiments, the plenoptic sensor is a nona-pixel sensor comprising a plurality of microlenses and a respective 3×3 array of color filter pixels under each microlens. The filter pixels have three different colors, and the colors of the color filter pixels are arranged such that each of the sub-aperture images generated from the plenoptic image has an extended Bayer pattern, and such that the pixels of a refocused image generated by adding the sub-aperture images with a disparity value of zero or one receive contributions from three pixels of the first color, three pixels of the second color, and three pixels of the third color.