3D Camera Timestamp Calibration with Epipolar Line Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D imaging technologies face challenges such as high power consumption, limited resolution at short distances, vulnerability to ambient light, and computational complexity, making them unsuitable for portable devices like smartphones for low-power, low-cost, and compact image sensors.
Innovation Solution
A low-power 3D imaging system using a CMOS image sensor with a laser point scan and timestamp calibration for depth measurements, which operates in both 2D and 3D modes, employing triangulation and epipolar geometry to reduce depth computations and power consumption, and allowing dual-use of the same image sensor for RGB imaging and 3D depth imaging.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If time-of-flight (TOF) method is used for 3D imaging, then depth measurement capability is achieved, but power consumption increases and device complexity increases
Solution Approach 1:
The patent replaces the active TOF depth sensing mechanism with a passive photogrammetry approach using existing camera images. By detecting vanishing points and epipolar lines from standard 2D images, the system infers 3D depth information without requiring additional active illumination or time-of-flight hardware, thereby reducing power consumption while maintaining depth measurement capability
Solution Approach 2:
The patent enables the existing camera system to serve dual purposes: standard 2D imaging and 3D depth measurement through photogrammetry. The same image sensor and processing unit are used for both functions, eliminating the need for separate TOF sensors or additional hardware components, thus reducing overall device complexity and power consumption
2Measurement precision
If structured light (SL) method is used for 3D imaging, then depth measurement capability is achieved, but device complexity increases and manufacturing cost increases
Solution Approach 1:
The patent extracts 3D depth information from standard 2D images by detecting geometric features such as vanishing points and epipolar lines. This approach removes the need for complex structured light projection systems, pattern generators, or specialized depth-sensing hardware, achieving depth measurement using only the existing camera and computational algorithms
Solution Approach 2:
The patent uses existing 2D images as copies or proxies for depth information. By analyzing the geometric relationships and perspective projections in standard images, the system reconstructs 3D spatial information without requiring dedicated depth-sensing hardware or structured light patterns, thereby simplifying device complexity and reducing manufacturing costs
3Measurement precision
If stereo vision system is used for 3D imaging, then depth measurement capability is achieved, but device complexity increases
Solution Approach 1:
The patent merges 3D depth measurement functionality with the existing single camera system. Instead of requiring separate stereo camera pairs or additional depth-sensing modules, the system combines photogrammetric algorithms with the existing image sensor to achieve depth measurement, thereby reducing device complexity while maintaining depth capability
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables efficient, low-power 3D depth measurements on portable devices like smartphones with improved resolution and reduced latency, while maintaining user safety and low power consumption, suitable for various applications including virtual reality and robotics.
Implementation Method 1
performing a one-dimensional (1D) point scan of a three-dimensional (3D) object along a scanning line using a light source, wherein the point scan projects a sequence of light spots on a surface of the 3D object
Implementation Method 2
for a pixel in the selected row of pixels, sensing a pixel-specific detection of a corresponding light spot in the sequence of light spots; (iv) in response to sensing the pixel-specific detection of the corresponding light spot, generating a timestamp value
Data Source
AI summary
Using the same image sensor to capture a two-dimensional (2D) image and three-dimensional (3D) depth measurements for a 3D object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor operates as a Time-to-Digital (TDC) converter to generate timestamps. A timestamp calibration circuit is provided on-board to record the propagation delay of each column of pixels in the pixel array and to provide necessary corrections to the timestamp values generated during 3D depth measurements.