Immersive Display Peripheral View Generation via Scene-Space Warping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing immersive display apparatuses face challenges in providing a wide view angle without increasing costs or reducing image resolution, as they often require multiple cameras or large-angle cameras, which can lead to excessive data processing and resolution loss.
Innovation Solution
An immersive display method and apparatus that processes a video signal with a narrow view angle by obtaining scene-space information, performing pre-warping and sampling neighborhood frames, excluding outliers, and blending the peripheral view image signal to maintain resolution and reduce processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If multiple cameras are used to photograph video at various angles, then the view angle is widened, but the cost increases and the amount of data to be processed increases excessively
Solution Approach 1:
The patent divides the video processing into multiple stages: pre-processing to obtain scene-space information, pre-warping to generate initial peripheral views, sampling to select key frames, warping to generate final peripheral views, and blending to combine with main-view video. This segmentation allows a single camera to achieve wide-angle immersion without requiring multiple cameras simultaneously
Solution Approach 2:
The patent creates virtual peripheral views by copying and transforming the main-view video signal through warping operations. Instead of using multiple physical cameras, the system generates synthetic peripheral views from the main video feed, maintaining cost-effectiveness while achieving the wide-angle effect
2Area of stationary object
If a camera with a large photographing angle is used, then the view angle is widened, but the image resolution is lowered
Solution Approach 1:
The patent transforms the 2D video signal into a 3D scene-space representation and then back to 2D peripheral views. By introducing the intermediate scene-space dimension with depth information, the system can generate high-resolution peripheral views without being constrained by the physical limitations of wide-angle camera lenses
Solution Approach 2:
The patent changes the processing approach from optical parameter changes (using wide-angle lenses) to computational parameter changes (applying warping transformations). By modifying the digital signal parameters through pre-warping and warping operations, the system achieves wide-angle views while preserving the original high resolution of the captured video
3Area of stationary object
If video signals are processed to generate wide view angle images, then the view angle is widened, but the processing time increases
Solution Approach 1:
The patent performs pre-processing and pre-warping operations in advance to obtain scene-space information and generate preliminary peripheral views. By preparing these intermediate results before the final warping and blending stages, the system reduces the computational burden during real-time processing, thereby reducing overall processing time
Solution Approach 2:
The patent applies sampling to select only key frames from the video sequence for warping operations. Instead of processing every frame, the system processes a subset of frames (excessive action on selected frames, partial action on others), which significantly reduces the total processing time while maintaining the illusion of continuous wide-angle video through frame interpolation and blending
Data Source
AI summary
The present disclosure relates to an immersive display apparatus and method for creation of a peripheral view image corresponding to an input video, the method comprising a pre-processing step of obtaining scene-space information at a main-view video signal corresponding to a first area, a pre-warping step of performing first warping to at least one neighborhood frame corresponding to a target frame included in the pro-processed video signal and determining an outlier from the result of the first warping, a sampling step of sampling at least one neighborhood frame to be used for extrapolation from the result of the first warping, a warping step of performing second warping to the sampled frame except for the outlier to generate a peripheral view image signal corresponding to a second area around the first area, and a blending step of blending the peripheral view image signal to the main-view video signal.