Neural Rendering Super-Resolution for 3D Scene Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for rendering three-dimensional (3D) scenes for virtual reality (VR) and augmented reality (AR) are time-consuming, labor-intensive, and costly, while neural rendering techniques still produce images of insufficient quality and accuracy.
Innovation Solution
A processor-implemented method that generates a first rendered image at a lower resolution by inputting target position information into a first model for a target object in a 3D scene, determines reference images from a set of images captured from different viewpoints, and then generates a second rendered image at a higher resolution by inputting the first rendered image, reference image, and position information into a second model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If 3D modeling technique is used to generate rendered images for 3D scenes, then the quality and accuracy of the rendered images can be maintained, but the time consumption and cost increase significantly
Solution Approach 1:
The system performs preliminary actions by capturing multiple images of the target object from different viewpoints in advance and storing them in a database. When a rendering request comes, the system quickly retrieves and synthesizes the pre-captured images using neural rendering techniques, avoiding time-consuming 3D modeling while maintaining high rendering quality.
2Productivity
If neural rendering technique is used to reduce modeling costs and time, then the efficiency improves, but the quality and accuracy of the generated images become insufficient
Solution Approach 1:
The system merges multiple low-resolution images captured from different viewpoints into a single high-resolution rendered image using neural rendering techniques. By combining information from multiple images and applying super-resolution algorithms, the system achieves both high rendering efficiency and high image quality simultaneously.
Solution Approach 2:
The system changes the resolution parameter dynamically during the rendering process. It starts with lower-resolution images for efficient processing and then applies super-resolution techniques to enhance the final output to high resolution, thus achieving both speed and quality.
3Manufacturing precision
If high-resolution images are generated directly using neural rendering, then the image quality is high, but the computational resources and time required increase
Solution Approach 1:
Instead of processing one high-resolution image directly, the system performs partial actions by processing multiple lower-resolution images first, then combines them to achieve the final high-resolution result. This distributes the computational load and reduces the energy required for each individual processing step.
Data Source
AI summary
A processor-implemented method includes generating a first rendered image having a first resolution by inputting target position information for a target viewpoint into a first model for a target object comprised in a three-dimensional (3D) scene, determining one or more reference images for the first rendered image from among a plurality of first images, wherein the plurality of first images is generated based on a plurality of original images in which the target object is captured from a plurality of different viewpoints, each of the plurality of original images has a second resolution, and each of the plurality of first images has the first resolution lower than the second resolution, and generating a second rendered image having the second resolution by inputting the first rendered image, the reference image, the target position information, and reference position information for the reference image into a second model.