Depth Mapping Using Visual Inertial Odometry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing depth mapping systems using time-of-flight sensing face challenges with high noise and low resolution, and are susceptible to motion artifacts due to relative motion between objects and the depth mapping apparatus, which complicates accurate depth measurement.
Innovation Solution
The system employs ancillary information from inertial sensors and additional image processing to detect and compensate for relative motion, adjusting histograms and extending exposure time when stationary, while filtering out motion artifacts, and using guidance filters to enhance depth map precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exposure time is extended to improve depth mapping precision, then measurement precision is improved, but motion artifacts increase due to relative motion during the extended exposure period
Solution Approach 1:
The system performs preliminary motion detection using inertial sensors and image processing before depth mapping. By detecting relative motion in advance and compensating for it through histogram adjustment, the system can extend exposure time to improve precision without suffering from motion artifacts. This preliminary action enables the system to prepare compensation strategies before the actual depth measurement process.
2Measurement precision
If motion compensation filtering is applied to reduce motion artifacts, then measurement precision is improved, but processing time increases due to additional filtering operations
Solution Approach 1:
The system applies motion compensation filtering selectively rather than uniformly across all depth data. By identifying regions affected by motion artifacts through image processing and applying filtering only to those specific areas, the system reduces overall processing time while still improving measurement precision where needed. This partial action approach avoids the computational overhead of processing entire depth maps when motion artifacts are localized.
3Measurement precision
If relative motion detection is performed using image processing to compensate for motion, then measurement precision is improved, but device complexity increases due to additional sensors and processing requirements
Solution Approach 1:
The system uses the image sensor to serve multiple functions: capturing color images for visual information and detecting relative motion through image processing. By making the image sensor multi-functional, the system reduces the need for separate dedicated motion detection sensors, thereby improving depth mapping accuracy without proportionally increasing device complexity. The same hardware resource performs dual roles.
Solution Approach 2:
The system introduces histogram data as an intermediary representation that bridges the gap between raw depth measurements and motion-compensated results. By processing and adjusting histograms rather than directly manipulating raw depth maps, the system simplifies the complexity of motion compensation operations. The histogram serves as an intermediate data structure that makes motion artifacts more manageable and easier to correct.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach improves the signal/noise ratio and resolution of depth maps by effectively reducing motion artifacts and enhancing the accuracy of depth measurements, even in varying ambient light conditions.
Implementation Method 1
measurement of the round-trip time, i.e. time of flight (ToF), taken by the optical beams as they travel from the source to the target scene and back to a detector array
Implementation Method 2
Objective optics are configured to form a first image of the target scene on the array of sensing elements
Data Source
AI summary
Imaging apparatus (22) includes a radiation source (40), which emits pulsed beams (42) of optical radiation toward a target scene (24). An array (52) of sensing elements outputs signals indicative of respective times of incidence of photons on the sensing elements. Objective optics (54) form a first image of the target scene on the array of sensing elements. An image sensor (64) captures a second image of the target scene. Processing and control circuitry (56, 58) is configured to process the second image so as to detect a relative motion between at least one object in the target scene and the apparatus, and which is configured to construct, responsively to the signals from the array, histograms of the times of incidence of the photons on the sensing elements and to adjust the histograms responsively to the detected relative motion, and to generate a depth map of the target scene based on the adjusted histograms.