Depth Correction Using Epipolar Contour Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The stereo measurement method for estimating depth in mixed reality struggles with accurate three-dimensional position and depth estimation in locations with few features, and is further compromised by positional shifts between image capturing devices due to aging or temperature changes, leading to significant depth errors.
Innovation Solution
An information processing apparatus that extracts contours from images, matches corresponding points, calculates depth, assesses reliability based on the angle between the epipolar line and contour tangential direction, and corrects depth errors using a reliability-based correction mechanism, even after calibration shifts occur.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If stereo measurement method is used for depth estimation, then depth information can be acquired from images, but accurate three-dimensional position and depth cannot be estimated in locations with few features because it is difficult to detect corresponding points
Solution Approach 1:
The patent applies infrared light pattern projection to apply features to an object, enabling corresponding point detection in locations with few visible features. The infrared patterns create artificial features that can be reliably detected and matched between stereo images, solving the problem of insufficient natural features for depth estimation.
Solution Approach 2:
The patent introduces infrared light patterns as an intermediary to facilitate corresponding point detection. These projected patterns serve as a mediator that provides detectable features on objects, enabling the stereo measurement system to acquire depth information even in feature-poor environments.
2Measurement precision
If contour matching is performed between image capturing devices, then depth measurement can be performed on objects with few features, but a large depth error can be generated depending on the shape of the contour
Solution Approach 1:
The patent uses infrared light pattern projection to create artificial features with specific geometric shapes (lines, circles, grids) that have well-defined mathematical properties. These structured patterns provide reliable corresponding points for contour matching while minimizing depth errors, as their known geometry allows for more accurate stereo triangulation regardless of the object's natural contour shape.
3Measurement precision
If calibration is performed on image capturing devices, then camera parameters can be estimated, but correct depth cannot be acquired when relative position and orientation shift due to aging or temperature changes
Solution Approach 1:
The patent performs preliminary calibration of camera parameters and stores them for later use in depth calculations. By establishing the baseline camera parameters in advance through careful calibration, the system can maintain depth measurement reliability even when minor shifts occur due to aging or temperature changes, as the calibrated parameters serve as a stable reference framework.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach effectively suppresses incorrect depth calculations and ensures accurate depth estimation in mixed reality applications by accounting for positional shifts, reducing depth errors and maintaining reliability across image capturing devices.
Implementation Method 1
the depth is estimated by the principle of triangulation based on the camera parameters and the corresponding points of the captured images
Data Source
AI summary
This invention provides an information processing apparatus which comprises an image acquiring unit configured to acquire images captured by a plurality of image capturing devices, a contour extracting unit configured to extract a contour of an object from the image acquired by the image capturing device, a matching unit configured to match corresponding points on the contours between the images, a depth calculating unit configured to calculate a depth of the matched point, a reliability extracting unit configured to acquire a reliability of the depth based on an angle formed by an epipolar line and a tangential direction of the contour, and a depth correcting unit configured to correct the depth based on the depth and the reliability.