Self-Annotated Smeared Point Removal in Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to effectively remove smeared points from depth maps, which are caused by multi-path reflections and occur at object boundaries, leading to false surfaces and degraded 3D reconstructions.
Innovation Solution
A fully automated annotation technique using geometric consistency across multi-frame data collections, combined with a pixel-level discriminator, to create a self-annotated smeared point detector. This method leverages dense depth sensing and multi-view consistency to automatically annotate points without the need for manual annotations or complex optical sensing models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If statistical outlier removal methods are used, then random noise is reduced, but actual surface points are also removed and smeared points remain
Solution Approach 1:
The patent segments the depth map processing into multiple stages: initial outlier removal, smeared point detection using multi-frame consistency analysis, and final refinement. This segmentation allows different processing strategies to be applied to different types of errors without interfering with each other, preserving surface points while removing smeared points.
Solution Approach 2:
The patent introduces an intermediary classification mechanism that analyzes temporal consistency across multiple frames to distinguish smeared points from valid surface points. This intermediary analysis layer processes depth information before final output, using multi-frame correlation as a mediator to identify and remove only the smeared points while preserving genuine surface features.
2Measurement precision
If hand-crafted filters are used to reduce noise, then general noise is reduced, but smeared points are not effectively removed and surfaces become overly smoothed
Solution Approach 1:
The patent employs dynamic, adaptive filtering that adjusts its strength based on local confidence measures derived from multi-frame consistency. Rather than applying uniform smoothing, the filter dynamically modulates its intensity across different regions of the depth map, applying stronger filtering only where smeared points are detected while preserving surface details in high-confidence regions.
Solution Approach 2:
The patent changes the parameters of the filtering operation based on temporal consistency analysis. The filter strength, kernel size, and application regions are all dynamically adjusted according to the measured consistency of each pixel across multiple frames, allowing selective removal of smeared points while maintaining surface detail integrity.
3Measurement precision
If multiple cameras or multiple frequencies are used, then smeared point detection improves, but acquisition overhead and complexity increase
Solution Approach 1:
The patent uses periodic temporal sampling of the scene using a single depth camera, capturing multiple frames over time. This periodic acquisition strategy provides the multi-view consistency information needed to detect smeared points without requiring multiple simultaneous cameras or frequency modulations, thereby reducing hardware complexity while maintaining detection accuracy.
Solution Approach 2:
The patent creates virtual multi-view copies of the scene by capturing it from the same spatial position at different time instances. These temporal copies serve the same analytical purpose as spatial copies from multiple cameras, allowing smeared point detection through temporal consistency analysis rather than requiring complex multi-camera hardware setups.
4Measurement precision
If synthetic datasets are used for training, then ground truth is available, but the models struggle to generalize to real-world scenarios
Solution Approach 1:
The system performs self-service by automatically generating its own training data through unsupervised temporal consistency analysis. The multi-frame processing pipeline inherently identifies smeared points without requiring manual annotation or synthetic dataset generation, allowing the system to learn directly from real-world data distribution and improve its performance on real scenarios while maintaining the benefits of having labeled training examples.
Data Source
AI summary
A method and system for removing smeared pixels from an image includes obtaining a plurality of training images of a scene from different poses, forming a point cloud of the scene having a plurality of pixels and a depth, and rendering a first pixel in a first reference frame to a second reference frame. The method includes comparing a depth difference of the first depth and the second depth, determining whether the pixel is valid or smeared based on the depth difference, associating a label with the pixel corresponding to valid or smeared, training a classifier with the pixel and the label to form a trained classifier, obtaining an image to be classified at the classifier and classifying the pixels in the image as valid or smeared and removing smeared pixels from the image to form a cleaned image.