CMOS Filter Arrays for Multispectral, Multi-Polarization Color Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Optical cameras struggle to detect wavelengths and polarizations outside their filter array spectrum, leading to computational intensity and latency in obstacle recognition, particularly for difficult-to-perceive colors like orange, which affects navigation and obstacle avoidance in autonomous vehicles.
Innovation Solution
Implementing a multispectral, multi-polarization filter array in CMOS image sensors that captures and processes data from multiple filter types, including different polarizations, to enhance depth estimation and object detection by combining data from overlapping cameras using image signal processors and machine learning algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a typical filter array with mosaic of filters tuned to capture particular wavelength features is used, then the camera can capture information of specific spectral features, but it cannot detect wavelengths spectrally outside the filter array spectrum, leading to computational intensity and latency in obstacle recognition
Solution Approach 1:
The filter array is segmented into multiple distinct filter types (first filter type and second filter type) with different spectral characteristics. This segmentation allows the system to capture diverse spectral information simultaneously, enabling detection of both standard and difficult-to-perceive colors without requiring a single complex filter design.
Solution Approach 2:
The filter array is designed with multi-functionality by incorporating filters that capture both conventional spectral information and specialized spectral features (such as orange wavelength detection). This universal design enables the same camera system to handle various obstacle recognition scenarios without requiring separate specialized sensors.
2Loss of information
If the filter array is designed to capture specific wavelength ranges, then it can provide targeted spectral information, but it struggles to detect wavelengths and polarizations outside its filter array spectrum, affecting navigation and obstacle avoidance
Solution Approach 1:
The filter array uses a composite structure combining different filter types (first filter type and second filter type) with complementary spectral responses. This composite approach ensures broader spectral coverage and more reliable obstacle detection by capturing information across multiple wavelength ranges simultaneously, reducing information loss for various obstacle types.
3Adaptability or versatility
If multiple filter types are combined in the filter array, then the system can capture diverse spectral information, but the processing of data from overlapping cameras becomes computationally intensive
Solution Approach 1:
The system performs preliminary spectral filtering at the sensor level by incorporating multiple filter types directly in the filter array. This preliminary action captures diverse spectral information in a structured format during image acquisition, reducing the computational burden during later processing stages compared to attempting to analyze unfiltered broadband data.
4Loss of time
If the camera system uses a standard filter array, then it can maintain simple device architecture, but it experiences latency in recognizing obstacles having colors outside the filter's spectral response
Solution Approach 1:
The filter array segments the spectral detection task across multiple filter types, with each filter type optimized for specific wavelength ranges. This segmentation enables simultaneous capture of various color information, reducing the time required to detect obstacles with colors outside the spectral response of a single filter type while maintaining detection accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Improves the ability of vehicles to accurately perceive and navigate through environments by enhancing the detection of challenging colors and features, reducing latency and increasing the efficiency of obstacle recognition.
Implementation Method 1
a first polarizer of the polarizer combination may have a first polarization axis oriented at a first angle relative to a horizontal axis of the image sensor, and a second polarizer of the polarizer combination may have a second polarization axis oriented at a second angle relative to the horizontal axis of the image sensor
Implementation Method 2
Each filter array may include a mosaic of filters with a feature that is responsive to particular properties of incident light (e.g., wavelength or polarization)
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In one embodiment, a method includes accessing first image data generated by a first image sensor having a first filter array that has a first filter pattern. The first filter pattern includes a first filter type corresponding to a spectrum of interest and a second filter type. The method also includes accessing second image data generated by a second image sensor having a second filter array that has a second filter pattern different from the first filter pattern. The second filter pattern includes a number of second filter types, the number of second filter types and the number of first filter types have at least one filter type in common. The method also includes determining a correspondence between one or more first pixels of the first image data and one or more second pixels of the second image data.