Light Field Occlusion Removal via Depth-Weighted Pixel Blending
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current light field image processing technologies fail to effectively remove occlusions in surveillance images, which hampers facial recognition accuracy in crowded scenes, as refocusing methods often inadequately handle occluding objects.
Innovation Solution
A method involving a light field camera system with depth sensors that captures multiple views, generates depth maps, projects pixels to calculate virtual distances, and blends them using weightings to generate a refocused target view, effectively removing occlusions by synchronizing color or grayscale values from multiple camera views.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If refocusing technology is used to process light field images, then the ability to see through occluders is improved, but occluding objects are not adequately removed or blurred for sufficient identification
Solution Approach 1:
The patent segments the light field data by capturing multiple camera views and separating pixels based on their depth information. Each pixel is projected to multiple camera views and weighted according to depth comparisons, effectively segmenting occluded regions from non-occluded regions. This segmentation enables selective processing where occluding objects are removed while preserving the occluded target.
Solution Approach 2:
The patent utilizes the fourth dimension of light field data (spatial-angular information) by capturing images from multiple viewpoints simultaneously. By projecting pixels to multiple camera views and using depth information from a different dimensional perspective, the system can identify and remove occlusions that appear in 2D images but are resolvable in the 4D light field space.
2Measurement precision
If multiple camera views are captured and processed to remove occlusions, then facial recognition accuracy is improved, but processing time and system complexity increase
Solution Approach 1:
The patent performs preliminary actions by pre-calculating depth maps for each camera view before the actual occlusion removal process. Depth information is obtained in advance using depth sensors or stereo matching, and pixels are pre-projected to multiple camera views. This preliminary processing organizes the data structure so that the final occlusion removal and blending operations can be performed efficiently in real-time.
3Reliability
If depth maps are generated for each camera view to enable occlusion removal, then the ability to identify occluded objects is improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent implements a universal depth estimation approach that can work with different sensor configurations. The same depth map generation pipeline can accommodate various depth sensors (time-of-flight, structured light, stereo cameras) or even pure computational stereo matching. This multi-functionality reduces the need for specialized hardware for each application scenario, managing system complexity while maintaining effectiveness.
Data Source
AI summary
A method of image processing for occlusion removal in images and videos captured by light field camera systems. The method comprises: capturing a plurality of camera views using a plurality of cameras; capturing a plurality of depth maps using a plurality of depth sensors; generating a depth map for each camera view; calculating a target view on a focal plane corresponding to a virtual camera; set a weighting function on the pixels on the camera views based on the depth map and a virtual distance; and blending the pixels in accordance with the weighting function to generate a refocused target view.