Plenoptic Depth Map Generation via Variance Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Plenoptic cameras face challenges in generating high-quality depth maps due to occlusions, which result in low resolution and visible artifacts around thin objects and sharp edges, especially when capturing images with a sparse set of views.
Innovation Solution
A method that calculates the variance in pixel intensities across multiple views, masks out depth planes with high variance, and iteratively recalculates to determine accurate depth layers, using Gaussian distributions and edge masking to resolve occlusions and improve depth estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If plenoptic cameras use a microlens array to capture 4D radiance, then the ability to refocus after picture taking is improved, but the image resolution deteriorates to low resolution with visible artifacts
Solution Approach 1:
The patent segments the depth information by iteratively separating pixels into different depth layers based on variance analysis. This segmentation allows the system to handle different depth planes independently, improving resolution by processing each layer separately rather than treating the entire image as a single depth plane.
Solution Approach 2:
The patent transitions from 2D image processing to 4D light field processing by incorporating depth as an additional dimension. Through iterative variance analysis and depth plane separation, the system extracts depth information and processes pixels in multiple dimensional spaces, enabling high-resolution refocusing without sacrificing the refocusing capability.
2Device complexity
If plenoptic cameras capture images with a sparse set of views, then the device complexity is reduced, but the measurement precision of depth deteriorates due to occlusions
Solution Approach 1:
The patent implements feedback through iterative variance calculation and depth plane masking. The system calculates variance, identifies occluded regions, masks them out, and recalculates variance in subsequent iterations. This feedback loop continuously improves depth estimation accuracy by eliminating the influence of occlusions from previous iterations.
Solution Approach 2:
The system uses the captured light field data itself to resolve occlusions through self-service mechanisms. By analyzing variance patterns within the captured data and iteratively masking problematic depth planes, the system extracts accurate depth information without requiring additional external inputs or more complex hardware configurations.
3Productivity
If traditional depth estimation methods are used on plenoptic images, then the processing speed is maintained, but the depth map quality deteriorates with visible artifacts at occlusion boundaries
Solution Approach 1:
The patent performs preliminary actions by calculating variance for all pixels before final depth assignment. This preliminary variance analysis identifies potential occlusion regions in advance, allowing the system to mask problematic areas before they contaminate the final depth map, thereby preventing artifact formation while maintaining processing efficiency.
Solution Approach 2:
The system skips problematic depth planes that contain occlusions by masking them out during iterative processing. Instead of spending computational resources trying to resolve ambiguous occluded regions, the method rapidly skips through iterations with masks applied, rushing through the problematic areas to preserve overall processing speed while improving depth map quality in non-occluded regions.
Data Source
AI summary
Described is systems and methods for determining the depth of pixels captured in a plenoptic image. The systems and methods may provide a plurality of views of each pixel location in the plenoptic image. One way of providing the plurality of views is to obtaining pixel intensities from the views for each pixel location in the plenoptic image. A variance can then be calculated of the distribution of pixel intensities for each pixel position. If the variance is below a predetermined threshold, the systems may mask out at least one depth plane and then recalculate the variance in the distribution of pixel intensities to determine if the variance is below the threshold.