Stereo Depth Estimation With Recursive Disparity Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining depth information using disparity maps from left and right eyes suffer from significant errors in regions difficult to match, such as repetitive, texture-less, or complex object edges, due to the isolated and hand-crafted nature of left-right consistency checks.
Innovation Solution
A recursive learning approach using a convolutional neural network and a ConvLSTM network to continuously correct disparity maps by leveraging complementary information from both eyes, incorporating attention maps to focus on and refine unreliable regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple images are captured at different positions and stereo vision is used to extract depth information, then depth data can be obtained, but the process requires complex image processing and multiple captures which reduces efficiency
Solution Approach 1:
The patent replaces the mechanical/optical system of multiple camera captures and stereo vision processing with a laser scanning system that directly measures depth through laser beam projection and reception. This substitution eliminates the need for complex image processing while achieving accurate depth information extraction.
Solution Approach 2:
The patent utilizes the phase transition concept in laser scanning by modulating the laser beam and measuring phase changes in the reflected light to determine depth. This allows direct depth measurement without requiring multiple images or complex processing, thereby improving efficiency while maintaining accuracy.
2Productivity
If laser scanning is used to acquire depth information, then processing time is reduced, but the system becomes sensitive to environmental light conditions
Solution Approach 1:
The patent applies local quality by using a structured light pattern (laser scanning) that creates a specific spatial distribution of light. This localized light pattern can be distinguished from ambient light, allowing the system to maintain high processing speed while being less sensitive to environmental lighting conditions.
Solution Approach 2:
The patent introduces an intermediary approach by using a laser scanner that projects structured light patterns as a mediator between the measurement system and the environment. This structured light acts as a unique identifier that can be reliably detected even in varying environmental light conditions, enabling fast processing without excessive sensitivity to ambient light.
3Measurement precision
If conventional image processing methods are used, then depth information can be extracted, but the methods are time-consuming and complex
Solution Approach 1:
The patent replaces the time-consuming image processing mechanism with a direct laser scanning measurement system. This substitution enables accurate depth data acquisition through optical measurement principles rather than computational image processing, significantly reducing processing time while maintaining or improving accuracy.
Solution Approach 2:
The patent applies preliminary action by pre-projecting structured light patterns onto the object surface before measurement. This prepares the scene in advance with known reference patterns, allowing for rapid depth extraction without requiring complex real-time image processing, thus reducing processing time while maintaining accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This method effectively reduces errors in depth information determination for challenging regions by iteratively improving disparity maps through recursive learning, enhancing accuracy in depth estimation for applications like self-driving vehicles and robotic systems.
Implementation Method 1
a laser scanner that scans an object with a laser beam and acquires depth information of the object
Data Source
Figure 1A~1B
Figure 2
Figure 3~4
AI summary
The present application discloses a method for determining depth information, comprising: acquiring a tth left-eye similarity matching from a left-eye image to a right-eye image and a tth right-eye similarity matching from the right-eye image to the left-eye image, wherein t is an integer greater than 1; performing, via a neural network model, processing on the tth left-eye similarity matching and a (t-1)th left-eye attention map so as to acquire a tth left-eye parallax image; performing, via the neural network model, processing on the tth right-eye similarity matching and a (t-1)th right-eye attention map so as to acquire a tth right-eye parallax image; and determining first depth information according to the tth left-eye parallax image, and determining second depth information according to the tth right-eye parallax image. The present application further discloses a device for determining depth information. The present application employs recursive learning to fully consider complementary information of two eyes so as to continuously correct a two-eye parallax image, such that for a region in which two-eye matching is difficult to perform, depth information errors can be effectively reduced.