ToF Depth Image Denoising With Edge-Preserving Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing denoising methods for time-of-flight (ToF) depth images, including neural networks, fail to effectively preserve edge information and adapt to dynamic noise distributions, leading to unsatisfactory performance and unwrapping errors due to noise and signal contamination.
Innovation Solution
A neural network architecture that utilizes both active brightness and complex depth data, incorporating an edge detecting layer to identify edges and perform convolutional operations, allowing for targeted denoising of non-edge pixels, and using varying kernel sizes based on edge proximity to mitigate noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If traditional denoising methods are applied to ToF depth images, then noise is reduced, but edge information is lost and unwrapping errors occur
Solution Approach 1:
The patent applies different processing strategies to different regions of the depth image based on edge detection results. Edge pixels are identified and preserved with minimal filtering, while non-edge pixels undergo more aggressive denoising. This local differentiation allows the system to reduce noise in smooth regions without blurring important edge features.
Solution Approach 2:
The patent segments the depth image into edge regions and non-edge regions using edge detection algorithms. By dividing the image into these distinct zones, the system can apply tailored denoising operations to each segment, preventing the uniform application of filters that would compromise edge integrity.
2Device complexity
If uniform denoising kernels are applied across the entire image, then processing is simple, but edge regions suffer from signal contamination and blurring
Solution Approach 1:
The patent employs adaptive kernel selection where the denoising kernel size and strength vary depending on the local region's characteristics. Edge regions utilize smaller or no kernels to preserve sharp transitions, while non-edge regions benefit from larger kernels that provide more aggressive noise reduction. This local adaptation optimizes both precision and processing efficiency.
3Object-affected harmful factors
If aggressive denoising is applied to reduce noise, then noise levels decrease, but edge sharpness and detail are degraded
Solution Approach 1:
The patent performs edge detection as a preliminary step before applying denoising operations. By identifying edge pixels in advance, the system can protect these critical regions from aggressive filtering while allowing more intensive denoising in non-edge areas. This preliminary classification enables subsequent processing to maintain edge sharpness while reducing noise effectively.
Data Source
AI summary
One example provides a computing device including a logic machine and a storage machine holding instructions executable by the logic machine to implement a depth image processing pipeline including a neural network, the neural network including an edge detecting layer. The neural network is configured to receive input of an active brightness image and receive input of one or more of real data or imaginary data of a complex depth image, the complex depth image corresponding to the active brightness image. The neural network is further configured to, at the edge detecting layer, apply one or more convolutional processes to the active brightness image to identify one or more edge pixels in the active brightness image, and at a second layer, denoise one or more of the real data or the imaginary data of the complex depth image based on the one or more edge pixels identified.