Mobile-Camera NLOS Tracking Across Multiple Planar Surfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing NLOS imaging methods are difficult to deploy in dynamic environments due to their reliance on precise optical alignment and specialized detectors, limiting their application in real-time tracking of hidden objects using moving cameras.
Innovation Solution
A data-driven framework, 'PathFinder', utilizing a vision transformer-based architecture that processes multiple planar surfaces with varying aspect ratios to enhance NLOS tracking performance, incorporating a preprocessing pipeline and a transformer network for real-time estimation of hidden object positions and velocities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing NLOS imaging methods use time-resolved detectors and laser configurations, then measurement precision is improved, but device complexity increases and ease of operation deteriorates
Solution Approach 1:
The patent replaces complex optical detection systems (time-resolved detectors and laser configurations) with a camera-based computational imaging approach. Instead of using specialized detectors requiring precise optical alignment, the system uses conventional cameras combined with computational algorithms to achieve NLOS imaging, thereby reducing device complexity while maintaining measurement precision
Solution Approach 2:
The patent changes the detection parameter from time-resolved measurements requiring precise optical alignment to image-based measurements that can be captured by conventional cameras. This parameter change enables NLOS imaging to be performed with simpler devices while operating in dynamic environments
2Measurement precision
If existing NLOS imaging methods use precise optical alignment, then measurement precision is improved, but ease of operation worsens in dynamic environments
Solution Approach 1:
The patent substitutes mechanical optical alignment requirements with computational image processing. By using camera images and computational algorithms instead of precisely aligned optical components, the system achieves NLOS tracking in dynamic environments without requiring manual optical alignment, significantly improving ease of operation
Solution Approach 2:
The patent enables dynamic NLOS tracking by using a moving camera platform that captures images from different positions and orientations. The computational framework processes these dynamic images to track hidden objects, allowing the system to operate in dynamic environments where the camera and objects are in motion, thereby improving ease of operation
3Device complexity
If conventional cameras are used for NLOS imaging, then device complexity is reduced, but measurement precision deteriorates due to low signal-to-noise ratio
Solution Approach 1:
The patent introduces computational algorithms as an intermediary between the conventional camera and the final NLOS image. The computational framework processes the low signal-to-noise ratio images captured by conventional cameras, enhancing the measurement precision through image processing and analysis without requiring complex hardware modifications
Solution Approach 2:
The patent creates multiple virtual copies of the NLOS scene from different camera positions and orientations. By capturing images from multiple viewpoints and computationally reconstructing the hidden object information, the system overcomes the low signal-to-noise ratio limitation of conventional cameras, improving measurement precision through redundant information gathering
Data Source
AI summary
Unlike existing passive methods that estimate an object's position based on a single stationary planar surface, a computer-implemented framework for non-line-of-sight (NLOS) imaging accommodates scenarios where a camera, steered by a robot platform, captures varying sections of multiple planar surfaces. The framework includes a data preprocessing pipeline for enhancing the signal-to-noise ratio (SNR) and facilitating scene understanding. Recognizing that all visible surfaces could contain valuable NLOS scatter information, the framework includes a transformer-based network that leverages captures from all of these surfaces of varying aspect ratios to estimate the position over time of a hidden NLOS object.