Plenoptic Camera Color Filter Array Shuffled Pattern
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Plenoptic cameras suffer from poor bokeh quality due to color artifacts during refocusing, especially when bokeh is present in images, and existing solutions require repositioning the color filter array from the sensor to the micro-lens or using random patterns, which are not feasible in all cases.
Innovation Solution
A color filter array with a pattern of cyan, yellow, yellow, and magenta (CYYM) or cyan, yellow, green, and magenta (CYGM) is positioned on the sensor, allowing for refocused images to be fully demosaiced without color artifacts by shifting colors based on specific coordinates, ensuring that the refocused images are free from Bayer pattern visibility and color artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a Bayer pattern color filter array is used on the sensor, then color images can be captured, but color artifacts appear in refocused images especially in bokeh regions
Solution Approach 1:
The patent applies asymmetry by using a non-uniform color filter distribution pattern (CYYM or CYGM) instead of the symmetric Bayer pattern. The color filters are strategically positioned at specific coordinates within each micro-image region, creating an asymmetric arrangement that enables artifact-free refocusing while maintaining color accuracy.
Solution Approach 2:
The patent segments the sensor surface into multiple micro-images, each corresponding to a specific sub-aperture. Within each micro-image, color filters are selectively applied only at certain pixel locations rather than uniformly across all pixels. This segmentation allows the refocusing algorithm to process regions with and without color filters differently, eliminating color artifacts in the final refocused image.
2Object-affected harmful factors
If the color filter array is repositioned from the sensor to the micro-lens, then color artifacts are eliminated, but the device complexity increases and feasibility is reduced
Solution Approach 1:
Instead of physically moving the color filter array from the sensor to the micro-lens, the patent creates a virtual copy of the color filter pattern in the computational domain. The CYYM or CYGM pattern is applied selectively to specific regions of the sensor data during image processing, replicating the effect of having color filters at the micro-lens without the mechanical complexity of repositioning physical filters.
Solution Approach 2:
The patent replaces the mechanical solution of physically repositioning color filters with a computational approach. By applying color filter patterns through digital image processing algorithms on the captured light field data, the system achieves the same color artifact elimination effect without modifying the physical sensor structure or requiring complex mechanical filter positioning mechanisms.
3Ease of operation
If a regular Bayer pattern is used, then demosaicing can be performed, but the Bayer pattern becomes visible in refocused images
Solution Approach 1:
The patent applies local quality by using different color filter configurations in different regions of the sensor. Specifically, CYYM or CYGM patterns are applied only to pixels that contribute to refocused images, while other regions maintain standard Bayer patterns. This localized differentiation allows the system to perform demosaicing where needed while preventing Bayer pattern visibility in refocused regions through the special color filter arrangement.
Data Source
Figure 1
Figure 2
Figure 3(a)~3(b)
AI summary
In one embodiment, it is proposed a plenoptic camera comprising a color filter array positioned on a pixel sensor, said color filter array comprising a set of unit elements, each unit element covering M × M pixels of said pixel sensor, with M an integer such that M ≥ 2, said plenoptic camera further comprising a set of micro-lens, each micro-lens delivering a micro-lens image on said pixel sensor with a diameter equal to p = k.M, with k an integer greater than or equal to one. The color filter array is remarkable in that said set comprises an initialization unit element being associated with a matrix cm,n0≤m<M0≤n<M indicating a filter repartition, where each coefficient cm,n is associated with a filter value, and in that the other unit elements are associated with matrixes with coefficients set to c(x+i)modM,(y+j)modM, for corresponding pixel (x, y, i, j) on said sensor, where indexes x, y relate to indexation of a pixel in said sensor, and indexes i, j relate to indexation of a micro-lens in micro-lens array.