Light Field Prediction Using Two-Plane Ray Parameterization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for predicting components of sub-aperture images in light field video coding, particularly with plenoptic cameras, are inefficient due to the assumption of a pinhole model, which does not accurately account for the extended aperture of plenoptic cameras, leading to suboptimal performance in light field video coding.
Innovation Solution
A method and device for predicting a component of a current pixel in a sub-aperture image captured by a type I plenoptic camera, which determines a location on the sensor based on the distance from the main lens to the micro-lens array, focal lengths, and camera model parameters to perform a two-plane parameterization of the field of rays, allowing for improved prediction using reference pixels from a neighborhood location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a pinhole model is used for light field video coding, then the coding process is simple, but the prediction accuracy deteriorates because the extended aperture of plenoptic cameras cannot be accounted for
Solution Approach 1:
The patent changes the mathematical model parameters from a simple pinhole model to a more complex model that incorporates the extended aperture characteristics of plenoptic cameras. This includes using a two-plane parameterization with specific camera parameters (distance D from exit pupil to micro-lens array, focal length F of main lens, focal length f of micro-lenses) to accurately describe the ray field corresponding to sensor pixels, thereby improving prediction accuracy while managing complexity through systematic parameterization
2Measurement precision
If camera-specific parameters are used for prediction, then prediction accuracy is improved, but the overhead increases
Solution Approach 1:
The patent performs preliminary action by determining the location on the sensor based on camera-specific parameters (distance D, focal length F, focal length f, and model parameters) before the actual prediction process. This two-plane parameterization is established in advance, allowing the prediction to proceed efficiently using pre-computed geometric relationships, thereby reducing the overhead during the main coding/decoding process while maintaining high prediction accuracy
Data Source
AI summary
Predicting a component of a current pixel belonging to a current sub-aperture image in a matrix of sub-aperture images captured by a sensor of a type I plenoptic camera can involve, first, determining a location on the sensor based on: a distance from an exit pupil of a main lens of the camera to a micro-lens array of the camera; a focal length of the main lens; a focal length of the micro-lenses of the micro-lens array; and a set of parameters of a model of the camera allowing for a derivation of a two-plane parameterization describing the field of rays corresponding to the pixels of the sensor; and, second, predicting the component based on one reference pixel belonging to a reference sub-aperture image in the matrix and located on the sensor in a neighborhood of the location.