Plenoptic MPI Encoding With Entropy-Based Depth Resolution Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face limitations in processing plenoptic multiplanar images (MPIs) for real-time video generation due to large data sizes and inefficient encoding methods, leading to constraints in uploading and rendering from multiple cameras and depth levels.
Innovation Solution
A device and method that calculates entropy for each depth level of MPI data, controls image resolutions based on entropy, and encodes level images using RGB channels, allowing for real-time processing of MPIs at 30 frames per second or more by packing and reducing resolutions adaptively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If MPI data is loaded in advance to main memory for real-time rendering, then image generation speed is improved, but memory capacity requirements increase significantly
Solution Approach 1:
The patent segments the large MPI data into multiple level images corresponding to different depth levels. Instead of loading the entire MPI data (e.g., 16 cameras × 1920×1080×32 depth levels×4 channels ≈ 4GB), the system divides it into manageable level images that can be processed and rendered independently, reducing the memory burden while maintaining real-time rendering capability.
2Productivity
If video frames are uploaded to graphics card memory, then real-time video processing is enabled, but the number of simultaneously processable frames is limited by memory size
Solution Approach 1:
The patent segments video frames into multiple level images based on depth levels. This allows the graphics card to process level images independently and in parallel, increasing the effective throughput. By processing segmented level images rather than complete frames, the system can handle more video data within the same memory constraints.
Solution Approach 2:
The patent dynamically adjusts the processing strategy by calculating entropy for each depth level and adaptively controlling the resolution of level images. This dynamic adaptation allows the system to optimize memory usage and processing efficiency based on the actual content complexity of different depth levels, enabling higher video processing throughput.
3Manufacturing precision
If MPI encoding is performed by separately encoding RGB and alpha parts, then encoding completeness is improved, but the number of generated video files increases excessively
Solution Approach 1:
The patent merges the encoding of RGB and alpha parts by packing level images using alpha channels to perform unified RGB encoding. This integration reduces the number of separate encoding operations and resulting video files, while still maintaining complete encoding information for both color and transparency data.
Solution Approach 2:
The patent introduces entropy calculation as an additional dimension for optimizing the encoding process. By calculating entropy for each depth level and using it to adaptively control resolution, the system achieves efficient compression and reduces the number of output files while preserving encoding completeness.
4Measurement precision
If all level images are maintained at original resolution, then image quality is preserved, but processing time and memory usage increase
Solution Approach 1:
The patent applies local quality by calculating entropy for each depth level and adaptively controlling the resolution of individual level images based on their specific entropy values. High-entropy level images (containing more information) are maintained at higher resolutions, while low-entropy level images are processed at lower resolutions. This localized quality adjustment preserves overall image quality while significantly reducing processing time and memory usage.
Data Source
AI summary
Provided are a device and method for processing a plenoptic multiplanar image (MPI). The device includes a video input part configured to receive plenoptic MPI data and a processor configured to calculate an entropy for each depth level of the plenoptic MPI data, control resolutions of level images according to the entropies, and encode the level images.