Spatially Varying Blur Extrapolates Incomplete 3D Image Regions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image generation methods for virtual reality applications face challenges in generating images from incomplete 3D data, leading to noticeable degradation and a disrupted immersive experience when users change viewpoints, especially in omnidirectional or immersive video scenarios.
Innovation Solution
An apparatus and method that extrapolates image content from complete regions to incomplete regions using a spatially varying filter to blur transitions, reducing perceptual impact and guiding users away from low-quality viewpoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If view point shifting processing is used to generate images for different view directions from incomplete 3D data, then image coverage for all positions and directions is improved, but image quality degradation becomes noticeable and immersive experience is disrupted
Solution Approach 1:
The patent converts the harmful effect of visible extrapolation artifacts into a beneficial immersive experience by intentionally introducing controlled blurring in transitional regions. This blurring masks the degradation caused by generating images from incomplete 3D data, making the artifacts imperceptible to users while maintaining full view coverage.
Solution Approach 2:
The patent applies different quality characteristics to different regions of the image. High-quality rendered regions are maintained where 3D data is available, while transitional regions use controlled blurring to mask degradation. This local differentiation allows the system to maintain overall image quality perception while providing complete view coverage.
2Adaptability or versatility
If dynamic virtual reality views are generated to match user movement and orientation changes, then immersive experience is improved, but computational complexity and processing requirements increase
Solution Approach 1:
The patent performs view point shifting and hole filling operations in advance to generate complete view images before they are needed for display. By pre-computing the extrapolated regions and applying blurring filters beforehand, the system reduces real-time computational complexity while maintaining dynamic immersive experience.
Solution Approach 2:
The patent introduces an intermediate processing stage that generates complete view images with controlled blurring before final rendering. This intermediary step separates the complex extrapolation operations from the real-time display requirements, reducing computational complexity during actual VR view generation.
3Manufacturing precision
If 9-26 view images are used for autostereoscopic displays instead of traditional two views, then display quality and depth perception are improved, but data requirements and processing load increase
Solution Approach 1:
The patent generates multiple view images for autostereoscopic displays by copying and transforming a limited set of input images through view point shifting. Instead of requiring 9-26 separate captured images, the system creates additional views by extrapolating from fewer source images, significantly reducing data requirements while maintaining display quality.
Solution Approach 2:
The patent makes the view point shifting algorithm universal, capable of generating any number of view images from a single set of input images. This multi-functional approach allows the same processing pipeline to serve both traditional stereo displays and advanced autostereoscopic displays with 9-26 views, reducing overall processing load.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus for generating an image comprises a receiver (101) which receives 3D image data providing an incomplete representation of a scene. A view vector source (103) provides a rendering view vector indicative of a rendering viewpoint for the image. A renderer (105) renders a first region of an intermediate image for the rendering view vector based on image data from the 3D image data. An extrapolator (107) extrapolates the 3D image data into a second adjoining region of the intermediate image with the 3D image not comprising image data for the second region. A blur processor (109) generates the image in response to applying a spatially varying blurring to the intermediate image where a degree of blurring is higher in a transitional region between the first region and the second region than in an internal area of the first region.