Depth Sensor Per-Pixel Calibration Non-Linear Error Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional depth sensor calibration techniques often result in limited accuracy, leading to incorrect distance measurements, which can be detrimental to applications such as 2D and 3D scene reconstruction, robot navigation, and manipulation, due to hardware capabilities and traditional calibration methods.
Innovation Solution
A method and system for correcting input depth images and calibrating depth sensors using a non-linear error model, where per-pixel correction values are determined and applied to compensate for systemic biases, utilizing a parameter matrix or lookup table for error correction, and patch-based calibration data to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional per-sensor calibration is used during manufacturing, then the calibration process is simple and fast, but the measurement precision of depth readings is limited
Solution Approach 1:
The calibration process is segmented into two distinct phases: (1) conventional per-sensor calibration during manufacturing for basic setup, and (2) a subsequent per-pixel calibration phase using captured depth images and reference measurements. This segmentation allows the system to achieve high measurement precision through the second phase while maintaining the simplicity and speed of the first phase during production.
Solution Approach 2:
The patent applies preliminary action by performing conventional calibration during manufacturing before deployment, establishing a baseline calibration state. Then, upon deployment, the system performs additional per-pixel calibration using captured depth images and reference distance measurements to achieve the desired high precision without requiring complex manufacturing processes.
2Measurement precision
If conventional calibration techniques are applied, then the calibration process is quick and easy, but depth sensors report incorrect distances compared to precision instruments
Solution Approach 1:
The system performs preliminary conventional calibration during manufacturing to establish basic operational parameters quickly. Then, upon deployment, it performs a more thorough per-pixel calibration using captured depth images and reference measurements from precision instruments, achieving accurate distance measurements while accepting the additional calibration time only when needed for high-precision applications.
Solution Approach 2:
The calibration system performs self-calibration by capturing its own depth images, comparing measurements against reference distances, and automatically generating per-pixel correction values without requiring external intervention or complex manual adjustment procedures.
3Measurement precision
If per-pixel correction is applied to correct depth readings, then measurement precision improves, but the complexity of the calibration system increases
Solution Approach 1:
The calibration data is segmented into per-pixel correction values stored in a structured format (parameter matrix or lookup table), separating the correction data from the conventional calibration parameters. This segmentation allows the system to maintain simple conventional calibration while adding sophisticated per-pixel corrections without overwhelming system complexity.
Solution Approach 2:
The patent introduces an intermediary calibration layer that sits between the conventional per-sensor calibration and the final depth readings. This intermediary per-pixel correction layer processes the depth values using correction values derived from captured images and reference measurements, achieving high precision without directly modifying the core sensor hardware or conventional calibration processes.
4Ease of manufacture
If conventional calibration is used, then manufacturing is simple and fast, but the depth sensor produces artifacts such as vertical bands in depth images
Solution Approach 1:
The calibration approach is segmented into conventional per-sensor calibration during manufacturing (maintaining simplicity) and per-pixel calibration after deployment (improving quality). This segmentation allows manufacturers to produce sensors quickly and simply while enabling end-users to achieve high-quality depth images without artifacts through the additional calibration step.
Solution Approach 2:
Conventional calibration is performed as a preliminary step during manufacturing to enable basic operation and keep the manufacturing process simple. The per-pixel calibration that eliminates artifacts is performed subsequently during deployment, allowing the system to achieve high reliability and clean depth images without complicating the manufacturing process.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Various technologies described herein pertain to correction of an input depth image captured by a depth sensor. The input depth image can include pixels, and the pixels can have respective depth values in the input depth image. Moreover, per-pixel correction values for the pixels can be determined utilizing depth calibration data for a non-linear error model calibrated for the depth sensor. The per-pixel correction values can be determined based on portions of the depth calibration data respectively corresponding to the pixels and the depth values. The per-pixel correction values can be applied to the depth values to generate a corrected depth image. Further, the corrected depth image can be output.