Light Field Rendering for See-Through Effects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current light field image rendering technologies struggle to effectively create see-through effects, particularly in security surveillance, as they often fail to adequately remove or blur occluding objects, limiting the identification and recognition of subjects behind occluders.
Innovation Solution
The proposed method employs a semantic light field rendering technique that assigns depth-guided and label-guided weights to each light field ray, using complex neural networks to differentiate objects by depth and category, and applies a semantics-consistent depth warping method to improve see-through effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional refocusing technology is used based on light field depth only, then the rendering process is simple, but the ability to remove or blur occluding objects is insufficient
Solution Approach 1:
The patent segments the light field rendering process into multiple independent modules: depth map generation, semantic segmentation, occluder identification, and weighted rendering. Each module handles a specific aspect of the problem, allowing complex functionality to be achieved through coordinated simple components rather than a monolithic complex system
Solution Approach 2:
The patent introduces intermediate data structures including depth maps, semantic segmentation maps, and occluder probability maps as mediators between the input light field and final rendered output. These intermediaries enable systematic processing of occluder removal while maintaining computational efficiency and system modularity
2Measurement precision
If only depth-based refocusing is applied, then processing speed is fast, but identification and recognition of occluded objects is limited
Solution Approach 1:
The patent performs preliminary actions by generating depth maps and semantic segmentation maps before the actual rendering process. Occluders are identified and marked in advance using trained neural networks, allowing the rendering stage to simply apply pre-computed weights rather than performing complex analysis during rendering
Solution Approach 2:
The patent replaces traditional mechanical/refocusing-based object identification with machine learning-based semantic segmentation and occluder detection. Trained neural networks automatically classify objects and identify occluders, achieving superior identification accuracy without increasing rendering time
3Reliability
If semantic segmentation and depth-guided weighting are combined, then see-through effects are enhanced, but computational complexity increases
Solution Approach 1:
The patent changes the parameter space by working with log-depth values and applying exponential weighting functions. This transformation simplifies the mathematical operations required for depth-guided rendering and allows efficient computation of weighted sums without requiring excessive computational resources
Solution Approach 2:
The patent uses pre-trained neural network models that can be reused across different rendering tasks. The depth estimation network and semantic segmentation network are trained once and then copied for use in multiple rendering scenarios, avoiding the need to retrain models and reducing computational energy consumption
Data Source
AI summary
A light field image processing method is disclosed for removing occluding foreground and blurring uninterested objects, by differentiating objects located at different depths of field and objects belonging to distinct categories, to create see-through effects. In various embodiments, the image processing method may blur a background object behind a specified object of interest. The image processing method may also at least partially remove from the rendered image any occluding object that may prevent a viewer from viewing the object of interest. The image processing method may further blur areas of the rendered image that represent an object in the light field other than the object of interest. The method includes steps of constructing a light field weight function comprising a depth component and a semantic component, where the weight function assigns a ray in the light field with a weight; and conducting light field rendering using the weight function.