3D Billboard Rendering Reduces Distortion and Compute Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating virtual views suffer from distortions and require excessive computational resources, making them impractical for common use, especially when generating a full 3D scene model is not feasible.
Innovation Solution
A method that involves capturing an image from a first perspective, segmenting it to extract a 2D contour, approximating 3D locations of points on the contour, generating a 3D billboard, and projecting the image onto it, using dynamic time warping to align contours from different camera perspectives, thereby reducing computational demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a full 3D scene model is created to generate virtual views, then the quality and accuracy of virtual views are improved, but the computational resources and input data requirements increase significantly
Solution Approach 1:
The patent segments the complex 3D modeling task by creating a simplified 3D billboard representation that captures only the essential geometric features needed for virtual view generation. This segmented approach extracts key contour points and fits simplified geometric primitives (planes, cylinders, spheres) to represent objects, avoiding the need for complete detailed 3D models while maintaining sufficient visual quality for virtual camera views.
Solution Approach 2:
The patent uses lightweight, computationally inexpensive 3D billboard representations instead of expensive full 3D scene models. These simplified geometric primitives serve as temporary substitutes that are sufficient for the specific purpose of generating virtual views from limited camera angles, without requiring the computational overhead of complete 3D scene reconstruction.
2Device complexity
If a planar billboard is used to generate virtual views, then the computational resources are reduced, but image distortions increase when viewed from angles significantly different from the capture angle
Solution Approach 1:
The patent extends the traditional planar billboard concept by allowing curved surfaces (cylinders and spheres) in addition to planes. This enables the billboard to bend and curve to match the actual 3D shape of objects, reducing geometric distortion when viewed from angles different from the capture angle while maintaining computational efficiency compared to full 3D modeling.
Solution Approach 2:
The patent transitions from 2D planar billboards to 3D curved billboards by adding depth and curvature dimensions. The billboard surfaces are defined in 3D space with curvature parameters, allowing them to wrap around objects and maintain accurate geometry when viewed from multiple angles, effectively adding spatial dimensions to the traditional flat billboard approach.
3Manufacturing precision
If traditional 3D modeling techniques are used to align contours from different camera perspectives, then the accuracy of virtual views is improved, but the computational time and resources increase
Solution Approach 1:
The patent extracts only the essential contour information needed for alignment without performing complete 3D modeling. It identifies key contour points from multiple camera views and directly fits simplified geometric primitives to these points, bypassing the time-consuming intermediate steps of full 3D scene reconstruction while maintaining sufficient alignment accuracy for virtual view generation.
Solution Approach 2:
The patent applies partial 3D modeling by creating simplified geometric representations (planes, cylinders, spheres) that capture the essential shape and orientation of objects without modeling all detailed features. This partial approach provides sufficient accuracy for contour alignment and virtual view generation while significantly reducing computational time compared to complete 3D modeling.
Data Source
AI summary
A method including receiving a first image of a scene captured from a first perspective, the first image including an object and a background; segmenting the first image to extract a first two-dimensional contour of the object; approximating a plurality of three-dimensional locations of a plurality of points on the first contour; generating a three-dimensional billboard of the object based on the three-dimensional locations; and projecting the first image onto the three-dimensional billboard.