Light Field Camera Occlusion Removal via Sub-Aperture Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cameras struggle to effectively remove occlusions from images, particularly when the foreground object attenuates or contributes unintended radiance, requiring manual intervention or multiple images, which is impractical for consumer photography and surveillance systems.
Innovation Solution
A method utilizing a light field camera to analyze sub-aperture images, separate foreground and background information, create a mask to reduce occlusion, and render a modified image, allowing for automated single-image occlusion removal without additional user input or multiple images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a conventional camera with a large aperture is used to see through thin occluding objects, then the background becomes visible through the occlusion, but the occluding object still contributes unintended radiance and attenuation to the image
Solution Approach 1:
The patent segments the image into foreground and background components by analyzing light field data from multiple sub-aperture images. By separating the radiance contributions from different depth planes, the system can identify and remove the foreground occluder's radiance while preserving the background scene information.
Solution Approach 2:
The patent extracts and removes the foreground occluder's radiance contribution from the composite image. By isolating the radiance coming from the occluding object and subtracting it from the total image radiance, the system recovers the background scene that was previously obscured or degraded by the foreground object's unintended radiance.
2Loss of information
If manual intervention or multiple images with specific settings are used to remove occlusions, then occlusion removal may be achieved, but the process becomes impractical for consumer photography and surveillance systems
Solution Approach 1:
The patent implements an automated occlusion removal system that processes images without requiring manual user intervention. The light field camera captures multi-view data, and the processing algorithm automatically identifies foreground occluders, separates their radiance contribution, and reconstructs the background scene autonomously, making the technology practical for consumer and surveillance applications.
Solution Approach 2:
The light field camera performs preliminary capture of multi-sub-aperture image data that contains encoded depth and radiance separation information. This preliminary data acquisition enables subsequent automated processing to remove occlusions without requiring the user to take multiple specially-configured images or perform manual adjustments at the time of image capture.
3Loss of information
If surveillance cameras are placed to avoid nearby occlusions, then clear scene coverage is achieved, but the placement options are restricted and may not provide optimal surveillance coverage
Solution Approach 1:
The patent converts the previously harmful effect of foreground occlusions into a beneficial capability. By using light field processing to remove occlusions, the system enables surveillance cameras to be placed in locations that would traditionally be avoided due to nearby objects. The occluding objects become irrelevant obstacles that the processing algorithm automatically eliminates, providing placement flexibility without sacrificing scene coverage quality.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables the removal of occlusions with significant reflectance or attenuation, improving image quality by focusing on the background while reducing the impact of foreground objects, suitable for hand-held camera captures and enhancing aesthetic and forensic utility.
Implementation Method 1
providing image information from a first image taken from a light field camera
Implementation Method 2
When focused on a distant scene, the large aperture induces a blur disc (e.g., the shape of the blur can be resultant of the shape of the aperture) at the occluders depth
Data Source
AI summary
Systems, methods, and devices for reducing an occlusion in an image are described herein. For example, one or more embodiments include a method including providing image information from a first image taken from a light field camera, analyzing image information from multiple sub-aperture images extracted from the image information of the first image to separate foreground and background appearance information, creating a mask of an occlusion based on the separated foreground and background appearance information, and rendering a second, modified image where the mask has been applied to reduce the occlusion in the modified image.