Photometric Stereo Object Detection in Autonomous Vehicle Cabins
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately detecting abandoned objects within their passenger cabins due to similarities in color and reflectivity between the vehicle interior and objects, leading to unreliable object recognition using single images, which existing methods struggle to address without requiring expensive hardware.
Innovation Solution
The system employs photometric stereo techniques using a camera to capture diversely-illuminated images, processing surface normals and albedo to create a normal-driven map, which is compared to a baseline map to detect and classify objects, allowing for reliable detection of abandoned items without additional hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple overlapping camera views or time of flight cameras are used to detect visually similar objects, then object detection accuracy is improved, but hardware cost and complexity increase
Solution Approach 1:
The patent changes the lighting parameters by capturing images under multiple different illumination conditions (different light sources, angles, and intensities). This allows the system to detect objects with similar colors by analyzing how they reflect light differently under varying conditions, achieving high detection accuracy without requiring expensive TOF cameras or multiple overlapping views.
Solution Approach 2:
The system uses periodic illumination changes by sequentially activating different light sources or adjusting lighting conditions across multiple image captures. This periodic variation in lighting enables the detection of subtle differences in object reflectivity and texture that are invisible in single static images, resolving the contradiction between using simple hardware and achieving high detection accuracy.
2Measurement precision
If multiple overlapping camera views or time of flight cameras are used to detect visually similar objects, then object detection accuracy is improved, but hardware cost increases
Solution Approach 1:
The patent achieves high detection accuracy by changing illumination parameters (light direction, intensity, color temperature) across multiple image captures using a single camera. This approach eliminates the need for expensive TOF cameras or multiple overlapping camera systems, thereby maintaining low hardware cost while achieving the ability to detect visually similar objects like a black laptop on a black leather seat.
Solution Approach 2:
The system creates multiple virtual views of the same scene by capturing images under different lighting conditions from a single camera position. These multiple illuminated images serve as copies that reveal different properties of objects, enabling accurate detection without requiring multiple physical cameras or expensive specialized hardware.
3Ease of manufacture
If single image object recognition is used, then hardware cost is reduced, but detection reliability deteriorates for visually similar objects
Solution Approach 1:
The patent maintains low hardware cost by using a single camera but improves reliability by changing illumination parameters across multiple image captures. The analysis of surface normals and reflectance properties under varying lighting conditions enables reliable detection of objects with similar colors and textures, resolving the contradiction between simple hardware and reliable detection.
Solution Approach 2:
The system employs periodic illumination changes by sequentially capturing images under different lighting conditions. This periodic variation in lighting parameters transforms the single-image recognition problem into a multi-temporal analysis, significantly improving detection reliability for visually similar objects while keeping hardware simple and inexpensive.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables accurate and reliable detection of abandoned objects using inexpensive hardware, improving detection accuracy for objects with similar color and reflectivity to the vehicle interior, such as thin objects, by leveraging photometric stereo image analysis and ambient lighting conditions.
Implementation Method 1
A camera captures image data including a plurality of diversely-illuminated images of a target area within a passenger cabin of the vehicle
Implementation Method 2
A normal extractor receives the images to determine a plurality of normal vectors for respective pixels representing the target area
Data Source
AI summary
Abandoned articles left by a user departing from an autonomous vehicle are automatically detected by capturing image data including a plurality of diversely-illuminated images of a target area within a passenger cabin of the vehicle. A plurality of normal vectors are determined for respective pixels representing the target area in a normal extractor based on the images. A normal-driven map is stored in a first array in response to the plurality of normal vectors. A baseline map is stored in a second array compiled from baseline images of the target area in a nominal clean state. Differences between the normal-driven map and the baseline map indicative of an object not present in the clean state are detected in a comparator. Difficult to detect objects can be found using a single, fixed camera.