Light-Field Image Fusion Using Consensus and Visibility Cubes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of managing and reducing the large volume of data generated by high-quality light-field image and video capture, which requires significant storage and computational resources, while maintaining image quality.
Innovation Solution
A method for processing volumetric data by selecting a subset of significant tiles from a virtual color cube representation, using consensus and visibility cubes to create a virtual color cube, and converting it into an atlas of tiles, which includes a residual image to maintain image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If light-field cameras capture rich three-dimensional scene information from various viewpoints, then image quality and information content are improved, but data volume increases significantly requiring more storage and computational resources
Solution Approach 1:
The light-field data is segmented into multiple virtual views or sub-aperture images representing different viewpoints. This segmentation allows selective processing and compression of individual view components, reducing overall data volume while preserving the ability to reconstruct high-quality multi-view images when needed.
Solution Approach 2:
Instead of storing complete high-resolution light-field data from all viewpoints, the patent creates compressed representations or proxies (such as depth maps, sparse viewpoint samples, or parameterized scene models) that can be used to generate or approximate the full light-field data on-demand, significantly reducing storage requirements while maintaining information quality.
2Device complexity
If conventional cameras capture two-dimensional images on sensors, then device complexity is reduced, but three-dimensional scene information and directionality are lost
Solution Approach 1:
The patent extends conventional 2D image capture into the third dimension by incorporating depth information through techniques such as time-of-flight sensing, structured light, or multi-view stereo reconstruction. This adds the Z-dimension to traditional XY image planes, enabling 3D scene understanding while building upon conventional camera architectures.
Solution Approach 2:
The patent introduces intermediate processing layers or computational models that bridge conventional 2D image data and 3D scene understanding. These intermediates (such as depth maps, normal maps, or scene graphs) serve as mediators that encode directional and three-dimensional information derived from multiple 2D views or active sensing, allowing standard cameras to achieve 3D capabilities through computational processing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method and system are provided for processing image content. In one embodiment the method comprises receiving a plurality of captured contents showing same scene as captured by one or more cameras having a different focal length and depth maps and generating a consensus cube by obtaining depth map estimations from said received contents. The visibility of different objects in then analysed to create a soft visibility cube that provides visibility information about each content. A color cube is then generated by using information from the consensus and soft visibility cube. The color cube is then used to combine different received contents and generate a single image for the plurality of contents received.