Camera-Based NLoS Obstacle Detection Using Shadow Registration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for non-line-of-sight (NLoS) obstacle detection in autonomous vehicles rely on expensive hardware infrastructure and are prone to interference from unpredictable lighting sources, limiting their effectiveness in occluded scenarios.
Innovation Solution
A method and system for NLoS obstacle detection using image capture devices to capture sequences of images, register them to a projected viewpoint, enhance images through color amplification, and classify them to issue control signals, leveraging shadow information to detect dynamic obstacles even when occluded.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If UWB systems or Wi-Fi signals are used for NLoS perception, then localization capability is improved, but hardware cost and system complexity increase
Solution Approach 1:
The patent extracts and utilizes only the necessary visual information from standard camera images, specifically focusing on shadow regions and their temporal changes. This approach eliminates the need for specialized UWB or Wi-Fi hardware while maintaining NLoS detection capability through careful analysis of existing visual data
Solution Approach 2:
The system uses the vehicle's existing camera infrastructure to perform NLoS detection, making the standard perception hardware serve multiple functions including both normal scene understanding and occluded obstacle detection through shadow analysis
2Measurement precision
If time-of-flight cameras are used to recover hidden scenes, then NLoS detection capability is improved, but reliability decreases due to interference from unpredictable lighting sources
Solution Approach 1:
The patent converts the harmful effect of unpredictable lighting sources into a beneficial detection mechanism by focusing on shadows cast by these very light sources. Instead of being disturbed by lighting variations, the system uses them to create detectable shadow patterns that reveal occluded obstacles
Solution Approach 2:
The system detects changes in illumination and shadow characteristics in the visual data, analyzing temporal variations in light and dark regions to identify moving occluded obstacles without being affected by the underlying lighting source variations
3Loss of information
If drones or UAVs are deployed for NLoS perception, then situational awareness is improved, but device complexity and deployment cost increase significantly
Solution Approach 1:
The patent makes the standard camera system universally capable of both normal driving perception and NLoS obstacle detection. By analyzing shadow dynamics in the visual stream, the same hardware performs multiple functions without requiring specialized drones or additional sensing infrastructure
Solution Approach 2:
The system extracts valuable NLoS information directly from the visual data stream without adding external sensing platforms. By focusing computational analysis on shadow regions and their temporal changes, it retrieves occluded obstacle information using only the existing camera system
4Measurement precision
If shadow information is used for NLoS detection, then detection capability in occluded scenarios is improved, but sensitivity to lighting condition changes increases
Solution Approach 1:
The system performs preliminary analysis of shadow patterns and their temporal changes before making detection decisions. By continuously monitoring and comparing shadow characteristics across multiple frames, it builds a baseline understanding that helps distinguish actual obstacles from normal lighting variations
Solution Approach 2:
The patent embraces lighting condition changes as the very mechanism for detection. Rather than trying to eliminate their effect, the system uses the dynamic shadow patterns created by varying lighting to reveal the presence and motion of occluded obstacles
Data Source
AI summary
A method of non-line-of-sight (NLoS) obstacle detection for an ego vehicle is described. The method includes capturing a sequence of images over a period with an image capture device. The method also includes storing the sequence of images in a cyclic buffer. The method further includes registering each image in the cyclic buffer to a projected image. The method includes performing the registering by estimating a homography H for each frame of the sequence of images to project to a view point of a first frame in the sequence of images and remove motion of the ego vehicle in the projected image. The method also includes enhancing the projected image. The method further includes classifying the projected image based on a scene determination. The method also includes issuing a control signal to the vehicle upon classifying the projected image.