Multilayered Point-Based 3D Rendering Soft Edge Preservation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image-based rendering techniques fail to preserve soft edges of objects in 3D views, leading to artifacts like 'halo' effects and jagged edges, and require high computation time due to many-to-one mapping and image matting issues.
Innovation Solution
A method and system that generate a multilayered point-based 3D representation by constructing an alpha model for regions with similar depth values, identifying mixed pixel regions, and estimating color, depth, and transparency values using associated image and depth information, allowing for accurate rendering of soft edges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional depth image based rendering is used, then rendering speed is maintained, but soft edges of objects cannot be preserved and artifacts like halo effects appear
Solution Approach 1:
The patent segments the depth map into multiple depth layers by clustering depth values. This segmentation allows separate processing of different depth regions, enabling accurate edge detection and soft edge preservation without halo artifacts. Each depth layer is processed independently to identify mixed pixels at object boundaries.
Solution Approach 2:
The patent applies different processing strategies to different regions of the image. For mixed pixels at object edges, it uses alpha matting and transparency computation. For regular pixels, it uses standard depth-based rendering. This local quality approach preserves soft edges while maintaining overall rendering efficiency.
2Manufacturing precision
If image matting is used to compute transparency values for edge pixels, then soft edges can be preserved, but computation time increases significantly
Solution Approach 1:
The patent segments the image into foreground, background, and mixed pixel regions using depth map analysis. By identifying only mixed pixels that require transparency computation, it avoids applying computationally intensive image matting to the entire image, thus reducing overall computation time while preserving soft edges where needed.
Solution Approach 2:
The patent applies transparency computation only partially - specifically to mixed pixels at object boundaries identified through depth map analysis. This partial action approach computes transparency values only where necessary for soft edge preservation, rather than processing the entire image, significantly reducing computation time.
3Device complexity
If one-to-one pixel matching between original image and depth map is enforced, then rendering simplicity is maintained, but many-to-one mapping at object edges cannot be handled correctly
Solution Approach 1:
The patent segments pixels into three categories: foreground pixels, background pixels, and mixed pixels. This segmentation allows the system to handle many-to-one mapping correctly for mixed pixels at edges while maintaining simple one-to-one mapping for regular pixels, thus balancing complexity and accuracy.
Solution Approach 2:
The patent applies different pixel matching strategies locally: simple one-to-one mapping for most pixels, and specialized many-to-one handling with transparency computation only for mixed pixels at object edges. This local differentiation maintains rendering simplicity for the majority of pixels while achieving accurate edge rendering where needed.
Data Source
AI summary
The present invention provides a method and system of generating multilayered point based 3D representation of a scene. The method includes constructing an alpha model of randomly selected regions associated with objects with similar depth value and depth jump in a depth map of a scene, and identifying mixed pixel regions associated with edges of the objects. The method further includes estimating color, depth and transparency values for pixels in each identified mixed pixel region using associated color information from an image map of the scene, associated depth value information from the depth map, and transparency information from the alpha model based on the associated depth information. The method includes generating a multi-layered point based 3D representation of the scene by organizing the estimated color, transparency, and depth values. Moreover, the method includes rendering 3D views of the scene using the multi-layered point based 3D representation of the scene.