Layered Scene Decomposition Codec for Real-Time Light Field Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in achieving real-time data transmission and efficient encoding/decoding for multi-dimensional content at light field displays, due to high bandwidth requirements and latency issues.
Innovation Solution
The method involves deconstructing 3D scene data into layers, sampling, and rendering these layers to compress data for transmission, and then decoding to construct and merge light fields at the display, using a layered scene decomposition (LSD) codec system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the entire light field data set is processed to reduce redundancy, then compression ratio is improved, but encoding and decoding processing time increases excessively
Solution Approach 1:
The patent divides the entire light field data set into multiple subcomponents (elemental images) that can be processed independently. Each subcomponent is encoded and decoded separately, allowing parallel processing and significantly reducing the time required compared to processing the entire data set as a single unit. This segmentation enables real-time encoding and decoding while maintaining compression efficiency.
2Reliability
If large light field data sets are read from storage medium, then complete data availability is improved, but transmission latency increases excessively
Solution Approach 1:
The patent segments large light field data sets into smaller subcomponents that can be read and processed in parallel from storage media. This allows multiple smaller data chunks to be retrieved simultaneously, significantly reducing total read time and transmission latency compared to reading the entire large data set sequentially. The segmented approach maintains data availability while enabling real-time processing.
3Loss of information
If standard video coding techniques are applied to light field subcomponents, then compression is achieved, but redundancy exploitation is insufficient
Solution Approach 1:
The patent creates a universal coding framework that integrates multiple coding techniques (transform coding, predictive coding, motion compensation) into a single system that can handle light field data. This multi-functional approach allows the system to exploit temporal redundancy between frames and spatial redundancy between elemental images simultaneously, achieving superior compression efficiency compared to standard video coding techniques applied to individual subcomponents.
Data Source
AI summary
A system and methods for a CODEC driving a real-time light field display for multi-dimensional video streaming, interactive gaming and other light field display applications is provided applying a layered scene decomposition strategy. Multi-dimensional scene data is divided into a plurality of data layers of increasing depths as the distance between a given layer and the display surface increases. Data layers which are sampled using an effective resolution function to determine a suitable sampling rate and rendered using hybrid rendering, such as perspective and oblique rendering, to encode light fields corresponding to each data layer. The resulting compressed, (layered) core representation of the multi-dimensional scene data is produced at predictable rates, reconstructed and merged at the light field display in real-time by applying view synthesis protocols, including edge adaptive interpolation, to reconstruct pixel arrays in stages (e.g. columns then rows) from reference elemental images.