Multichannel Multi-Polarization Imaging for Obstacle Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optical cameras with CMOS image sensors and filter arrays face challenges in detecting features outside their primary spectral response, leading to latency in recognizing obstacles, especially those with wavelengths like orange, which can be difficult to detect due to low spectral response, affecting navigation and obstacle avoidance in applications like autonomous vehicles.
Innovation Solution
The implementation of a multispectral, multi-polarization imaging system that combines data from multiple cameras with different filter arrays, allowing for the estimation of depth and object detection by interpolating data across different filter types, including those outside the primary response range, using a machine-learning algorithm and image signal processor to generate composite data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single camera with a standard filter array is used, then the device complexity is low, but the detection precision for wavelengths outside the primary spectral response is insufficient
Solution Approach 1:
The patent combines data from multiple cameras with different filter arrays (e.g., RGB camera, panchromatic camera, polarization camera) to create a composite multispectral image. This merging approach enables detection across a broader spectral range including wavelengths outside the primary response of any single camera, thereby improving measurement precision without requiring a single complex multispectral sensor
Solution Approach 2:
The system uses multiple standard cameras each optimized for specific functions (color detection, luminance detection, polarization detection) rather than a single specialized multispectral camera. This multi-functionality approach allows the system to detect various spectral features using conventional devices, improving detection precision while keeping individual device components relatively simple
2Adaptability or versatility
If multiple cameras with different filter arrays are combined, then the detection capability for challenging spectral features is improved, but the device complexity increases
Solution Approach 1:
The system segments the spectral detection task across multiple cameras, with each camera responsible for detecting specific spectral bands or features (e.g., one camera for blue-green wavelengths, another for red wavelengths, another for polarization). This segmentation allows the system to achieve broad spectral adaptability using multiple simple cameras rather than one complex multispectral instrument
Solution Approach 2:
The patent introduces an image signal processor and machine learning algorithm as intermediaries to fuse data from multiple cameras with different filter arrays. These intermediaries reconcile the different data formats and spectral responses, enabling versatile detection capability while allowing the use of standard, relatively simple camera devices
3Measurement precision
If data from multiple cameras is processed to generate composite multispectral data, then the object detection accuracy is improved, but the processing time and computational complexity increase
Solution Approach 1:
The system performs preliminary processing of camera data including demosaicing, color correction, and alignment before fusion. By preparing the data in advance with standardized processing pipelines, the system reduces the computational burden during real-time object detection, thereby improving detection accuracy while minimizing additional processing time
Solution Approach 2:
The patent replaces complex mechanical or hardware-based spectral filtering systems with computational methods for generating multispectral data. By using machine learning algorithms and image processing techniques to synthesize multispectral information from standard cameras, the system achieves high detection accuracy while reducing hardware complexity and processing time compared to traditional multispectral imaging systems
Data Source
AI summary
In one embodiment, a method includes accessing first image data generated by a first image sensor having a first filter array that has a first filter pattern. The first filter pattern includes a number of first filter types. The method also includes accessing second image data generated by a second image sensor having a second filter array that has a second filter pattern different from the first filter pattern. The second filter pattern includes a number of second filter types, the number of second filter types and the number of first filter types have at least one filter type in common. The method also includes determining a correspondence between one or more first pixels of the first image data and one or more second pixels of the second image data based on a portion of the first image data associated with the filter type in common.