Cascaded Multi-Stage Disparity Estimation for Accuracy and Speed

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current computer vision technologies for disparity estimation face challenges in balancing accuracy and speed, with methods like local area matching being fast but inaccurate, global optimization being accurate but energy-intensive, and neural networks being computationally expensive and memory-intensive.

Innovation Solution

A disparity estimation system that performs cascaded multi-stage disparity processing using feature extraction networks to generate multiple disparity maps with successively increasing sizes, leveraging edge and image-self-based features for refinement, allowing for flexible accuracy and speed adaptation across different devices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If global optimization methods are used for disparity estimation, then measurement precision is improved, but use of energy and device complexity increase significantly

Engineering Contradiction:
Improvedisparity estimation accuracyVSAvoidcomputational energy consumption
Core Design Contradiction:
Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent segments the disparity estimation process into multiple stages with different resolution levels. The first stage processes images at a lower resolution to generate an initial disparity map quickly, while subsequent stages process at higher resolutions for refinement. This segmentation allows the system to achieve high measurement precision through multi-scale processing while reducing overall energy consumption by avoiding computationally intensive global optimization at all resolution levels simultaneously.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies preliminary action by performing disparity estimation at a lower resolution first before proceeding to higher resolutions. The first stage generates an initial disparity map at reduced resolution, which then serves as input for subsequent refinement stages. This preliminary low-resolution processing provides a good starting point that reduces the computational burden in later stages, effectively balancing precision requirements with energy consumption.

Inventive Principle:
Principle #10Preliminary action

2Measurement precision

If neural network methods are used for disparity estimation, then measurement precision is improved, but device complexity and memory requirements increase

Engineering Contradiction:
Improvedisparity estimation accuracyVSAvoidsystem computational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the neural network processing into multiple stages with varying computational complexity. The first stage uses a simpler neural network architecture processing lower-resolution images, while subsequent stages use more complex architectures for higher-resolution refinement. This segmentation enables the system to achieve high measurement precision through progressive refinement while managing device complexity by not deploying the most complex network at all stages simultaneously.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies partial action by using different levels of network complexity appropriate to each stage's requirements. Rather than consistently using the most complex neural network architecture, the system uses simpler networks for initial low-resolution processing and progressively applies more complex networks only when higher precision is needed at higher resolutions. This partial application of computational resources balances measurement precision with device complexity.

Inventive Principle:
Principle #16Partial or excessive action

3Productivity

If local area matching methods are used for disparity estimation, then productivity is improved, but measurement precision deteriorates

Engineering Contradiction:
Improvedisparity estimation speedVSAvoiddisparity estimation accuracy
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The patent segments the disparity estimation process into multiple resolution stages, where the first stage uses local area matching methods at reduced resolution to achieve high productivity and fast processing. Subsequent stages then perform refinement at higher resolutions to improve measurement precision. This segmentation allows the system to benefit from the speed of local matching methods while still achieving the accuracy of more comprehensive methods through multi-scale processing.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies preliminary action by performing fast local area matching at lower resolution first to generate an initial disparity map quickly. This preliminary processing achieves high productivity and provides a good initial result that can be refined in subsequent stages. By doing the computationally cheap low-resolution matching first, the system achieves fast processing speeds while still having the opportunity to improve precision through later refinement stages.

Inventive Principle:
Principle #10Preliminary action

4Device complexity

If single-resolution disparity processing is used, then device complexity is reduced, but adaptability to different performance requirements decreases

Engineering Contradiction:
Improveprocessing architecture simplicityVSAvoidaccuracy-speed adaptation flexibility
Core Design Contradiction:
Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent implements universality by creating a multi-stage disparity processing system that can serve multiple functions and performance requirements simultaneously. The system generates disparity maps at multiple resolutions, allowing it to adapt to different application needs - from real-time low-precision requirements to high-precision requirements. This multi-functional architecture provides adaptability to various performance requirements while maintaining a unified processing framework that manages complexity effectively.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent applies dynamics by making the processing architecture adaptable through multiple stages that can be activated based on performance requirements. The system dynamically adjusts the resolution and computational intensity of processing stages depending on the specific application needs. This dynamic multi-stage design provides versatility for different accuracy-speed trade-off requirements while managing device complexity through a structured hierarchical approach.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentEP3836083B1Disparity estimation system and method, electronic device and computer program product
Publication Date: 2023.08.09 NEXTVPU (SHANGHAI) CO LTD
  • EP3836083B1 patent drawingFigure 1~2(c)
  • EP3836083B1 patent drawingFigure 3~4(b)
  • EP3836083B1 patent drawingFigure 5

AI summary

A disparity estimation system and method, an electronic device, and a computer program product are disclosed. The disparity estimation system includes: a feature extraction network configured to perform feature extraction on each image in an image pair and output extracted image features to a disparity generation network; and the disparity generation network configured to perform cascaded multi-stage disparity processing according to extracted image features to obtain multiple disparity maps with successively increasing sizes. The input of a first stage disparity processing in the multi-stage disparity processing includes multiple image features having a size corresponding to the first stage disparity processing, and the input of disparity processing of each stage other than the first stage disparity processing in the multi-stage disparity processing includes: one or more image features having a size corresponding to disparity processing of the stage and a disparity map generated by disparity processing of an immediate previous stage.