Monocular Depth Estimation Using Phase-Pixel Image Rearrangement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional fixed-focus lenses in vehicle image systems struggle to accurately estimate the distance of objects outside the focal plane, leading to difficulties in high-resolution image capture and safe operation of autonomous vehicles.
Innovation Solution
A computer-implemented method and system that utilizes a sensor system with micro-lenses and phase-pixels to estimate depth and refine image edges, incorporating a-priori lens information and sensor gain ratios to generate transformed images for enhanced resolution and depth perception.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a fixed-focus lens is used in the vehicle image system, then the system reliability is improved by eliminating autofocus mechanisms, but the depth estimation accuracy deteriorates for objects outside the focal plane
Solution Approach 1:
The pixel array is segmented into multiple phase-pixels (e.g., 2x2 or 3x3 grids) within each pixel unit, with each phase-pixel capturing light from slightly different angular directions. This segmentation enables the system to measure phase differences and estimate depth for objects at various distances from the focal plane, resolving the depth estimation limitation of fixed-focus lenses while maintaining system reliability.
Solution Approach 2:
The invention adds a new dimension of measurement by utilizing the phase information from multiple phase-pixels within each pixel unit. Instead of relying solely on intensity information from a single focal plane, the system measures the phase shift of light across different sub-pixels, enabling depth estimation in the third dimension (distance from camera) without moving the lens.
2Measurement precision
If traditional phase detection with moving lens is used, then the depth estimation accuracy is improved, but the system reliability deteriorates due to potential failures of autofocus mechanisms
Solution Approach 1:
The invention replaces the mechanical autofocus system with an optical phase-detection system. Instead of physically moving the lens to achieve focus, the system uses the phase information captured by multiple phase-pixels to estimate depth. This substitution eliminates mechanical components that can fail while maintaining depth estimation capability through optical measurements.
3Device complexity
If fixed-focus lens is used, then the device complexity is reduced by eliminating autofocus mechanisms, but the image resolution deteriorates for objects outside the focal plane
Solution Approach 1:
By segmenting each pixel into multiple phase-pixels, the system can capture phase information from different angular directions simultaneously. This segmentation enables the reconstruction of high-resolution images for objects at various depths without requiring complex autofocus mechanisms, thus maintaining simple device architecture while improving image resolution across different focal planes.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Improves image resolution and depth estimation accuracy, enabling safer autonomous vehicle operations by distinguishing real objects from ghost images and enhancing perception systems.
Implementation Method 1
receiving image data captured by a sensor system of a vehicle and indicating an object in an object field of the vehicle, the sensor system including a plurality of micro-lenses and a pixel
Data Source
AI summary
A method includes receiving image data captured by a sensor system indicating an object in an object field of a vehicle, the sensor system including a plurality of micro-lenses and a pixel, each micro-lens of the plurality of micro-lenses corresponding to a sub-pixel of the pixel, each sub-pixel of the pixel having a plurality of phase-pixels. The method also includes identifying a respective phase ratio of the pixel, each of the sub-pixels of the pixel, and each phase-pixel of the plurality of phase-pixels, and identifying, based on the respective phase ratios of each of the phase-pixels, a depth of the object in the object field. The method also includes estimating an edge of the object, rearranging the sub-pixels of the pixel to generate a transformed image file of the object, and generating, for output to a viewing stack and a perception stack, the transformed image file.