3D Scene Decoding with Gradient-Weighted Point Cloud Blending
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for decoding depth and texture image data from projected 3D scenes suffer from imprecision and ghosting effects due to rasterization and non-lossless compression, leading to inaccuracies in reconstructing colored point clouds, especially when viewed from different camera orientations.
Innovation Solution
A method involving de-projection of pixels from source images based on depth and color attributes, with a scoring system that weights blending of color attributes according to local depth and color gradients, to generate a viewport image that reduces ghosting effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If depth and texture image data are compressed using non-lossless compression methods, then data transmission efficiency is improved, but reconstruction precision of the colored point cloud deteriorates
Solution Approach 1:
The patent combines multiple projected images of the same 3D point cloud from different viewpoints into a single reconstructed point cloud. By merging information from multiple sources, the system compensates for losses introduced by compression in any single image, thereby maintaining reconstruction precision while still using efficient compression methods.
Solution Approach 2:
The patent creates multiple copies of the 3D scene from different projection viewpoints. Each copy contains redundant information about the same points, and by having multiple copies, the system can reconstruct the original point cloud more accurately even if individual copies have compression artifacts.
2Loss of information
If a same point in the original colored point cloud is inverse projected several times from different images, then more color information is available, but ghosting effects increase due to coordinate and color imprecisions
Solution Approach 1:
The patent implements a feedback mechanism where each reconstructed point is assigned a score based on the quality and reliability of its source pixels. The blending process uses these scores to weight contributions from different projections, automatically suppressing low-quality contributions that would cause ghosting while preserving high-quality color information.
Solution Approach 2:
The patent changes the parameter weighting in the blending process by introducing score-based weights. Instead of treating all inverse-projected points equally, the system dynamically adjusts their influence based on local depth and color gradient scores, thereby reducing ghosting effects from low-quality projections.
3Adaptability or versatility
If the rebuilt colored point cloud is captured with a virtual camera not at the center of projection, then viewing flexibility is improved, but ghosting artifacts increase due to accumulated errors
Solution Approach 1:
The patent performs preliminary error correction during the reconstruction phase by scoring and weighting points before the viewing stage. By addressing coordinate and color imprecisions during reconstruction rather than during rendering, the system enables flexible viewing without amplifying ghosting artifacts.
Data Source
AI summary
Generating an image from a source image can involve encoding a projection of a part of a three-dimensional scene. Pixels of a source image comprise a depth and a color attribute. Pixels of a source image are de-projected as colored point cloud. A de-projected point in a 3D space has the color attribute of the pixel that it has been de-projected from. Also, a score is attributed to the generated point according to a local depth gradient and/or a local color gradient of the pixel it comes from, the lower the gradient, the higher the score. The generated point cloud is captured by a virtual camera for rendering on a display device. The point cloud is projected onto the viewport image by blending color of points projected on a same pixel, the blending being weighted by the scores of these points.