Hybrid 2D-3D Sensor Array With Metalens and ROI Depth Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D imaging systems for applications like ADAS, autonomous driving, AR, and VR face challenges in providing high-resolution and high-quality 3D depth information due to hardware and software limitations, such as bulky form factors, alignment issues between 2D and 3D sensors, and high computational requirements, which hinder efficient object detection and tracking at longer distances.
Innovation Solution
A hybrid sensing system with a shared field of view (FOV) for 2D and 3D sensing regions, utilizing a metalens to direct visible and NIR light to respective sensing regions and a processing circuit to combine 2D and 3D information, along with a region of interest (ROI) scanning mechanism to conserve power and reduce computational strain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If separate detection systems are used for 2D imaging and 3D depth sensing, then detection capability is improved, but device complexity and alignment difficulty increase
Solution Approach 1:
The patent combines 2D imaging sensors and 3D depth sensing sensors into a single hybrid sensor array, where each pixel location contains both a photodiode for 2D imaging and a SPAD for 3D depth sensing. This integration eliminates the need for separate detection systems and their associated alignment procedures, while maintaining comprehensive detection capabilities for both 2D images and 3D depth information.
2Adaptability or versatility
If separate detection systems are used for 2D imaging and 3D depth sensing, then detection capability is improved, but alignment precision deteriorates
Solution Approach 1:
By merging 2D and 3D sensing capabilities into a single hybrid sensor array with co-located photodiodes and SPADs at each pixel position, the patent ensures perfect spatial alignment between 2D image data and 3D depth data. This eliminates alignment errors that occur with separate systems, as both sensing types share the exact same field of view and pixel coordinates.
3Measurement precision
If high resolution 3D depth information is provided, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The patent implements a region of interest (ROI) scanning mechanism that selectively activates only the necessary portion of the hybrid sensor array for 3D depth sensing. Instead of continuously activating the entire array, the system identifies ROIs based on 2D image content and directs 3D sensing resources only to those regions, significantly reducing overall power consumption while maintaining high-resolution depth information where needed.
4Measurement precision
If high resolution 3D depth information is provided, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent segments the sensor array into distinct functional regions: 2D imaging regions with photodiodes and 3D depth sensing regions with SPADs. This segmentation allows each sensor type to be optimized for its specific function while being integrated in a systematic pattern, reducing the overall complexity compared to a fully integrated multi-functional pixel design.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The hybrid sensing system enables high-resolution, high-quality 3D imaging with improved object detection and tracking capabilities at greater distances, reducing power consumption and computational demands while maintaining a compact form factor.
Implementation Method 1
detect ambient light reflected from an object
Implementation Method 2
a metalens on the hybrid sensing array, the metalens being configured to direct the ambient light reflected from the object towards the 2D sensing region, and to direct the light emitted by the light source and reflected from the object towards the 3D depth sensing region
Implementation Method 3
a 2D sensing region configured to detect ambient light reflected from an object and a 3D depth sensing region configured to detect the light emitted by the light source and reflected from the object
Data Source
AI summary
Provided is a 3D depth sensing system and method of providing an image based on a hybrid sensing array. The 3D sensing system including a light source configured to emit light, a hybrid sensing array comprising a 2D sensing region configured to detect ambient light reflected from an object and a 3D depth sensing region configured to detect the light emitted by the light source and reflected from the object, a metalens on the hybrid sensing array, the metalens being configured to direct the ambient light reflected from the object towards the 2D sensing region, and to direct the light emitted by the light source and reflected from the object towards the 3D depth sensing region, and a processing circuit configured to combine 2D image information provided by the 2D sensing region and 3D information provided by the 3D depth sensing region to generate a combined 3D image.