Predictive Light-Field Compression via Refocus Image Pools
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high-dimensional and irregularly sampled nature of light-field data makes it difficult to compress effectively using traditional image/video compression techniques, resulting in unsatisfactory results due to the lack of band-limited assumptions and spatial or frequency domain correlation.
Innovation Solution
The integration of refocus image construction into the compression and decompression processes, where a refocus image pool is maintained at different depths to predict light-field data, allowing for block-wise processing and improved predictive power, and the use of arbitrary four-dimensional sampling grids, which is compatible with common image/video compression standards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional image/video compression techniques are applied to light-field data, then compression is performed, but the compression quality is unsatisfactory due to high-dimensional and irregularly sampled nature of light-field data
Solution Approach 1:
The light-field data is divided into multiple depth layers or slices, allowing independent processing of each layer. This segmentation reduces the complexity of handling the entire 4D light-field at once while maintaining compression quality through layer-specific optimization.
Solution Approach 2:
The patent transforms the 4D light-field data into a different dimensional representation, such as converting angular and spatial dimensions into a structured format that resembles traditional 2D or 3D image data. This dimensional transformation enables the application of conventional compression techniques while preserving the unique properties of light-field data.
2Reliability
If light-field data is stored in full resolution, then all spatial and angular visual information is preserved, but the storage space required becomes orders of magnitude larger than 2D images
Solution Approach 1:
The patent extracts and separates the essential visual information from the redundant data in light-field representations. By identifying and removing duplicate or highly correlated angular samples, the method retains the core spatial and angular details while dramatically reducing storage requirements.
Solution Approach 2:
The patent transforms light-field data by changing its parameter representation, such as converting from raw 4D coordinates to a parameterized form that encodes depth, angle, and position more efficiently. This parameter transformation reduces the data volume while maintaining the ability to reconstruct high-quality refocused and panoramic views.
3Measurement precision
If refocus image pool is maintained at different depths for prediction, then predictive power is improved, but memory access pattern and bandwidth requirements increase
Solution Approach 1:
The patent pre-organizes the refocus image pool in memory according to depth layers and access patterns that are optimized for the prediction process. By preparing and sorting reference images in advance based on their depth and spatial characteristics, the system reduces the need for random memory access during compression, thereby lowering bandwidth requirements while maintaining prediction accuracy.
Data Source
AI summary
According to various embodiments, a light-field image may be compressed and/or decompressed to facilitate storage, transmission, or other functions related to the light-field image. A light-field image may be captured by a light-field image capture device having an image sensor and a microlens array. The light-field image may be received in a data store. A processor may generate a first refocus image pool with a plurality of refocus images based on the light-field image. The processor may further use the first refocus image pool to compress the light-field image to generate a bitstream, smaller than the light-field image, which is representative of the light-field image. The processor or a different processor may also be used to generate a second refocus image pool with a second plurality of images based on the bitstream. The second refocus image pool may be used to decompress the bitstream to generate a reconstructed light-field image.