Binocular Depth Estimation via Parallax and Confidence Segmentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Deep learning-based systems for binocular stereo matching often produce inaccurate predictions, particularly in critical areas like small obstacles, leading to failures and unreasonable results in various scenarios.

Innovation Solution

A method and apparatus for estimating the depth of a binocular image by inputting the image into a predetermined neural network to obtain a parallax image and a confidence score image, where the discrete parameter of each pixel in the parallax image has a negative correlation with its confidence score, allowing for error detection and improved accuracy through unsupervised learning.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If deep learning networks are used for binocular stereo matching, then prediction speed is improved, but prediction accuracy deteriorates in critical areas such as small obstacles

Engineering Contradiction:
Improveprediction speedVSAvoidprediction accuracy
Core Design Contradiction:
SpeedVSMeasurement precision

Solution Approach 1:

The patent segments the prediction output into two distinct components: a parallax image providing depth information and a confidence score image providing reliability information. This segmentation allows the system to process and evaluate different aspects of prediction quality separately, enabling improved accuracy in critical areas without sacrificing overall prediction speed.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The confidence score image serves as a feedback mechanism that indicates the reliability of each pixel's depth prediction. By providing this feedback information, the system can identify areas with low confidence (such as small obstacles) and apply corrective measures or further processing specifically to those regions, thereby improving prediction accuracy where it matters most while maintaining efficient processing elsewhere.

Inventive Principle:
Principle #23Feedback

2Device complexity

If deep learning networks assume accurate predictions, then system complexity is reduced, but reliability deteriorates due to undetected errors

Engineering Contradiction:
Improvesystem complexityVSAvoidsystem reliability
Core Design Contradiction:
Device complexityVSReliability

Solution Approach 1:

The system performs self-evaluation by generating confidence score images that automatically assess the reliability of its own predictions. This self-service mechanism allows the deep learning network to identify potential errors without requiring external validation or complex verification systems, thereby improving reliability while adding minimal complexity.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The confidence score image acts as an intermediary between the raw prediction and the final decision-making process. It provides a layer of evaluation that bridges the gap between the simple deep learning output and the need for reliable, error-free predictions, enabling the system to handle uncertainty without requiring fundamentally more complex architecture.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS11295466B2Method and apparatus for estimating depth of binocular image, device, program, and medium
Publication Date: 2022.04.05 SHENZHEN SENSETIME TECH CO LTD
  • US11295466B2 patent drawing
  • US11295466B2 patent drawing
  • US11295466B2 patent drawing

AI summary

A method and apparatus for estimating the depth of a binocular image, a device, a program, and a medium are provided. The method includes: inputting a to-be-processed binocular image into a predetermined neural network to obtain a parallax image and a confidence score image thereof, where a discrete parameter of each pixel in a preset distribution in the parallax image has a negative correlation with a confidence score of the pixel; and obtaining a depth image of the to-be-processed binocular image according to the parallax image.