Multi-View Image Processing With Precomputed Camera Warp Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing technologies struggle to efficiently create three or four dimensional renderings from two-dimensional images due to challenges in parallax estimation and depth map calculations, especially in multi-view content, which requires extensive data processing and storage, and current techniques do not effectively handle camera parameters, particularly when cameras are not horizontally aligned.
Innovation Solution
A method and system that utilize camera parameters, including intrinsic and extrinsic parameters, to calculate depth maps and simplify decoder calculations by pre-computing matrix products and warp maps, allowing for efficient transmission and rendering of multi-dimensional visual perceptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth maps are calculated using multiple cameras and parallax estimation, then three-dimensional visual perception is achieved, but the amount of data to be transmitted and processed increases significantly
Solution Approach 1:
The patent pre-calculates depth maps and camera parameters at the encoder side before transmission. By performing depth estimation and camera calibration in advance, the system avoids transmitting raw multi-view data that would require extensive processing at the decoder, thus reducing transmitted data volume while maintaining depth perception quality
Solution Approach 2:
The patent extracts and transmits only essential depth map information and pre-computed camera parameters rather than transmitting all raw multi-view image data. This selective extraction of critical depth-related information reduces the data burden while preserving the ability to render three-dimensional visual perception at the decoder
2Reliability
If multiple views with different viewing angles are provided to create motion parallax, then three-dimensional visual perception is improved, but storage and processing requirements become challenging
Solution Approach 1:
The patent performs camera calibration and computes camera parameters for multiple views in advance at the encoder side. By pre-processing the multi-view data to extract camera geometry information and depth maps, the system simplifies the decoder's task to merely rendering based on provided parameters, thus reducing processing complexity while maintaining visual perception quality
Solution Approach 2:
The patent introduces pre-computed depth maps and camera parameters as intermediary data structures that bridge the gap between raw multi-view images and final rendered output. These intermediaries encapsulate the complex geometric relationships between multiple cameras, allowing the decoder to efficiently render three-dimensional views without directly processing the complex multi-view data
3Adaptability or versatility
If camera parameters are not pre-computed, then flexibility in camera configuration is maintained, but decoder computational load increases
Solution Approach 1:
The patent pre-computes camera parameters including intrinsic and extrinsic parameters at the encoder side before transmission. By performing these computationally intensive calculations in advance, the system maintains flexibility in camera configuration (since parameters can represent any camera setup) while significantly reducing the decoder's computational load to merely applying the pre-computed parameters for rendering
Data Source
Figure 1
Figure 2a~2b
Figure 3
AI summary
A method and system are provided for processing image content. The method comprises receiving information about a content image captured at least by one camera. The content includes multi-view representation of an image including both distorted and undistorted areas. The camera parameters and image parameters are then obtained and used to determine to which areas are undistorted and which areas are distorted in said image. This is used to calculate depth map of the image using the determined undistorted and distorted information. A final stereoscopic image is then rendered that uses the distorted and undistorted areas and calculation of depth map.