Multiplane Image View Synthesis for Real-Time Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional photo-realistic rendering techniques require intensive manual and computational effort, limiting high-quality digital imagery creation to experts and demanding significant computational resources, making it difficult to achieve realistic images of real-world scenes in virtual experiences.
Innovation Solution
The system processes image data to render novel viewpoints by promoting input images to multiplane images, using a convolutional neural network to blend renderings from neighboring images, allowing for real-time photorealistic image generation on limited hardware, and providing interactive guidance for capturing images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional photo-realistic rendering techniques are used, then high-quality realistic images can be generated, but intensive manual and computational effort is required and significant computational resources are demanded
Solution Approach 1:
The patent uses image-based rendering that copies visual appearance from multiple captured images rather than constructing scenes from scratch using complex 3D models and ray tracing. The system captures multiple images from different viewpoints and uses these as direct references for generating novel views, eliminating the need for intensive manual scene construction and complex rendering computations while maintaining photorealistic quality
Solution Approach 2:
The patent segments the rendering process into distinct computational steps: capturing multiple images from different viewpoints, representing scenes as multiplane images with depth planes, and synthesizing novel views through weighted combinations of rendered target viewpoints. This segmentation allows each step to be optimized independently and reduces overall computational complexity compared to conventional monolithic rendering approaches
2Manufacturing precision
If conventional rendering techniques are used, then photorealistic images can be produced, but creation is limited to experts with substantial expertise
Solution Approach 1:
The patent enables self-service operation by allowing users to capture images using simple handheld devices and automatically generating 3D scene representations and novel views through computational processing. The system handles the complex tasks of multiplane image construction, depth plane generation, and view synthesis automatically without requiring user expertise in 3D modeling or rendering techniques, making the technology accessible to casual users
3Manufacturing precision
If high-resolution photorealistic rendering is performed, then realistic images are achieved, but substantial computing time on high-resource machines is required
Solution Approach 1:
The patent applies partial action by capturing images only at strategically selected viewpoints rather than requiring complete 360-degree coverage from all possible angles. The system uses a limited set of captured images and generates additional virtual viewpoints through computational synthesis, achieving high-quality rendering without the time cost of capturing every possible view or using extremely high-resolution processing throughout the entire pipeline
Data Source
AI summary
A respective target viewpoint may be rendered for each of a plurality of multiplane images of a three-dimensional scene. Each of the multiplane images may be associated with a respective single plane image of the three-dimensional scene captured from a respective viewpoint. Each of the multiplane images may include a respective plurality of depth planes. Each of the depth planes may include a respective plurality of pixels from the respective single plane image. Each of the pixels in the depth plane may be positioned at approximately the same distance from the respective viewpoint. A weighted combination of the target viewpoint renderings may be determined, where the sampling density of the single plane images is sufficiently high that the weighted combination satisfies the inequality in Equation (7). The weighted combination of the target viewpoint renderings may be transmitted as a novel viewpoint image.