Light Field Decoding with SAI Super-Resolution and Quality Enhancement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current encoding and decoding standards for light field pictures are inefficient and inaccurate, leading to high computational complexity and poor compression performance due to the unique characteristics of light field data from lenslet arrays.
Innovation Solution
Implement a method using a super-resolution reconstruction net on the decoding end for spatial and angular super-resolution reconstruction of low-resolution SAIs, combined with down-sampling on the encoding end to reduce spatial and angular resolution, and utilize a Quality Enhancement Net (QENet) to enhance image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current encoding and decoding standard is directly applied to light field picture, then processing simplicity is maintained, but processing accuracy and compression efficiency are poor
Solution Approach 1:
The patent segments the light field picture processing into multiple stages: initial SAI extraction from bitstream, super-resolution reconstruction to obtain reconstructed SAI, and quality enhancement through QENet to produce target SAI. This segmentation allows each stage to be optimized independently, improving overall processing accuracy while managing complexity through modular architecture.
Solution Approach 2:
The patent performs preliminary down-sampling processing on the light field picture at the encoding end before compression. This preliminary action reduces the data size and complexity that needs to be processed during encoding and decoding, while the super-resolution reconstruction at the decoding end restores the necessary quality, thus improving efficiency without sacrificing accuracy.
2Productivity
If indirect compression method converting light field picture to SAI is used, then compression is achieved, but computational complexity greatly increases and processing accuracy decreases
Solution Approach 1:
The patent applies partial down-sampling processing to the light field picture, focusing on reducing spatial and angular resolution in a controlled manner. This partial action achieves compression by retaining only the most essential information, while the super-resolution reconstruction recovers the necessary details, thus improving compression efficiency without requiring excessive computational resources for full-processing.
3Quantity of substance
If down-sampling processing is performed on SAI to obtain initial SAI, then transmitted bitstream size is reduced, but spatial and angular resolution decrease
Solution Approach 1:
The patent performs down-sampling as a preliminary action during encoding to reduce bitstream size. The reduced-resolution initial SAI is then compressed and transmitted. At the decoding end, super-resolution reconstruction is applied to recover the spatial and angular resolution, producing the target SAI with quality comparable to or exceeding the original. This preliminary compression followed by reconstruction resolves the contradiction between bitstream size and resolution.
Data Source
AI summary
A light field image processing method, a light field image encoder and decoder, and a storage medium are provided. The light field image processing method includes: a light field image decoder parsing a code stream, so as to obtain an initial sub-aperture image; inputting the initial sub-aperture image into a super-resolution reconstruction network, and outputting a reconstructed sub-aperture image, wherein the spatial resolution and the angular resolution of the reconstructed sub-aperture image are both greater than the spatial resolution and the angular resolution of the initial sub-aperture image; and inputting the reconstructed sub-aperture image into a quality enhancement network, and outputting a target sub-aperture image.