Distributed Depth Data Processing Pipeline
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current depth sensing systems, such as time-of-flight cameras, face challenges in accurately processing depth image data due to noise issues and computational intensity, particularly in low-power devices, where denoising prior to phase unwrapping can be resource-intensive and bandwidth-consuming.
Innovation Solution
A distributed depth engine pipeline is implemented, where pixel-wise phase unwrapping is performed locally on the camera, generating coarse depth images that are then sent to a remote computing system for more intensive denoising using convolutional filtering, reducing power consumption and bandwidth requirements while allowing larger denoising kernels for improved precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If denoising is performed prior to phase unwrapping on the camera, then depth image quality is improved, but power consumption and computational resource usage increase
Solution Approach 1:
The patent segments the depth processing pipeline into two distinct parts: lightweight operations (phase unwrapping) performed on the camera and heavy operations (denoising) performed remotely. This segmentation allows the system to perform necessary processing without consuming excessive power on the camera device itself.
Solution Approach 2:
The patent extracts the computationally intensive denoising operation from the camera's processing pipeline and relocates it to a remote computing system. This extraction reduces the computational burden and power consumption on the camera while maintaining the quality improvement benefits of denoising.
2Measurement precision
If denoising is performed prior to phase unwrapping on the camera, then depth image quality is improved, but data transmission bandwidth requirements increase
Solution Approach 1:
The patent performs phase unwrapping as a preliminary action on the camera before transmission to the remote system. By completing this computationally intensive operation locally first, the system transmits only the necessary unwrapped phase data rather than raw depth data, reducing the amount of data that requires transmission.
3Productivity
If phase unwrapping is performed after denoising, then processing sequence is conventional, but computational intensity and power consumption increase on the camera
Solution Approach 1:
The patent inverts the conventional processing sequence by performing phase unwrapping before denoising rather than after. This inversion allows the system to transmit smaller datasets to the remote system while still achieving the quality benefits of denoising, as the critical phase unwrapping operation is completed locally first.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces power consumption and increases processing efficiency by performing compute-intensive operations remotely, enabling higher precision in depth imaging while optimizing bandwidth usage and resource allocation.
Implementation Method 1
In ToF imaging, a distance to a point on an imaged surface in the environment is determined based on a length of a time interval in which light emitted by the ToF camera travels out to that point and then returns back to a sensor of the ToF camera.
Implementation Method 2
a light source on the TOF camera illuminates a scene with amplitude modulated light
Data Source
AI summary
Examples are provided that relate to processing depth camera data over a distributed computing system, where phase unwrapping is performed prior to denoising. One example provides a time-of-flight camera comprising a time-of-flight depth image sensor, a logic machine, a communication subsystem, and a storage machine holding instructions executable by the logic machine to process time-of-flight image data acquired by the time-of-flight depth image sensor by, prior to denoising, performing phase unwrapping pixel-wise on the time-of-flight image data to obtain coarse depth image data comprising depth values; and send the coarse depth image data and active brightness image data to a remote computing system via the communication subsystem for denoising.