AI Depth Map Unwrapping for Time-of-Flight Phase Ambiguity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Time-of-Flight (ToF) cameras face a distance ambiguity issue due to the wrapping of phase measurements into a fixed interval, limiting their unambiguous range and accuracy in depth sensing.
Innovation Solution
An electronic device equipped with artificial intelligence (AI) circuitry, specifically a convolutional neural network (CNN) of the U-Net type, is used to unwrap depth maps by learning patterns from training data, including iToF and RGB images, to determine wrapping indexes and resolve phase ambiguity, thereby extending the unambiguous range beyond the modulation frequency limits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If phase measurements are wrapped into a fixed interval for direct depth map determination, then the measurement process is simplified and processing speed is improved, but distance ambiguity occurs and unambiguous range is limited
Solution Approach 1:
The patent segments the depth measurement problem into two parts: first obtain a wrapped depth map quickly through phase measurement, then separately determine wrapping indexes through AI processing to resolve ambiguity. This segmentation allows the system to maintain fast initial processing while adding accuracy through subsequent unwrapping operations.
Solution Approach 2:
The patent introduces wrapping indexes as an intermediary element that bridges the wrapped phase measurements and the true unwrapped depth values. The AI algorithm processes the wrapped depth map to generate wrapping indexes, which then serve as a mediator to reconstruct the unambiguous depth information without requiring fundamental changes to the fast phase measurement process.
2Measurement precision
If multiple frequencies are used to extend unambiguous range, then distance measurement accuracy is improved, but processing time increases and frame rate decreases
Solution Approach 1:
The patent performs preliminary action by obtaining a wrapped depth map first using standard phase measurement, then uses AI processing to determine wrapping indexes and unwrap the depth map. This preliminary wrapped depth acquisition allows the system to avoid time-consuming multi-frequency measurements while still achieving extended unambiguous range through subsequent AI-based unwrapping.
Solution Approach 2:
The patent replaces the mechanical/multi-frequency measurement approach with an AI-based processing approach. Instead of physically measuring at multiple frequencies to extend range, the system uses a single frequency measurement followed by AI algorithms to computationally extend the unambiguous range, significantly reducing processing time.
3Device complexity
If conventional unwrapping algorithms are used, then computational resources are reduced, but accuracy in noisy conditions deteriorates
Solution Approach 1:
The patent changes the parameters processed by the AI algorithm beyond just the wrapped depth map, incorporating additional information such as amplitude data and spatial context. This parameter expansion allows the AI model to maintain high accuracy in noisy conditions while providing a unified framework that can adapt computational resources based on implementation.
Data Source
AI summary
An electronic device comprising circuitry configured to unwrap a depth map or phase image by an artificial intelligence algorithm to obtain an unwrapped depth map is disclosed. A main input is subject to denoising to obtain a pre-processed main input, such as a pre-processed depth map. An artificial intelligence process, e.g. a convolutional neural network such as CNN has been trained to determine wrapping indexes from main input and side information data. This artificial intelligence process is performed on the pre-processed main input and pre-processed side information to obtain respective wrapping indexes. A postprocessing, such as an unwrapping algorithm is performed based on the wrapping indexes to obtain an unwrapped depth map. The U-Net architecture is used in a specific type of segmentation task, in which the boundaries are not dictated by objects but by passing unambiguous range boundaries.