Parallel 3D Estimation via Local Hypothesis Propagation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current algorithms for 3D estimation from stereo images cannot be fully parallelized due to the dependency of 3D information calculation for a current pixel on the 3D information of preceding pixels, limiting computational efficiency and scalability in real-time applications.

Innovation Solution

The proposed method performs local 3D information estimation for each pixel using a small set of hypotheses from previous frames or iterations, allowing parallel processing on GPUs by randomly updating 3D information estimates based on statistical distributions, reducing the need for numerical updates from neighboring pixels and enabling efficient parallel computation across thousands of cores.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If 3D information calculation for a current pixel depends on 3D information of preceding pixels, then calculation accuracy is improved, but computational parallelization is limited

Engineering Contradiction:
Improve3D information calculation accuracyVSAvoidcomputational parallelization capability
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent segments the 3D information calculation into independent hypothesis evaluation units for each pixel. Instead of sequentially processing pixels with dependency on preceding pixels, each pixel's hypotheses are evaluated independently using local image data and propagated hypotheses from previous iterations, enabling parallel processing across thousands of GPU cores while maintaining calculation accuracy through iterative refinement.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary action by generating and propagating hypotheses from previous frames or iterations before final selection. Hypotheses are pre-computed and propagated to current pixel positions based on motion estimates and spatial coherence, allowing the current iteration to start with informed hypotheses rather than exhaustive search, thus enabling parallelization while preserving accuracy.

Inventive Principle:
Principle #10Preliminary action

2Measurement precision

If numerical updates from neighboring pixels are used, then 3D information accuracy is improved, but computational complexity increases

Engineering Contradiction:
Improve3D information accuracyVSAvoidcomputational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies partial action by using a small set of propagated hypotheses from neighboring pixels rather than comprehensive numerical updates from all neighbors. This selective approach uses only the most relevant hypotheses (e.g., from a limited spatial neighborhood or based on motion coherence) to inform current pixel estimation, reducing computational complexity while maintaining accuracy through the sufficiency of key neighbor information.

Inventive Principle:
Principle #16Partial or excessive action

Solution Approach 2:

The patent changes parameters by representing 3D information as a discrete set of hypotheses with associated confidence scores rather than continuous numerical values requiring complex optimization. This hypothesis-based representation transforms the problem from continuous parameter optimization to discrete hypothesis selection and propagation, significantly reducing computational complexity while preserving accuracy through the structured hypothesis space.

Inventive Principle:
Principle #35Parameter changes

3Speed

If parallel processing on GPUs is implemented, then processing speed is improved, but memory bandwidth requirements increase

Engineering Contradiction:
Improveprocessing speedVSAvoidmemory bandwidth requirements
Core Design Contradiction:
SpeedVSQuantity of substance

Solution Approach 1:

The patent segments the computation into independent pixel-level hypothesis evaluation tasks that can be executed in parallel on GPU cores. Each pixel's hypothesis evaluation uses only local image data and propagated hypotheses, minimizing the need for global memory access. This segmentation allows efficient use of GPU parallelism while reducing memory bandwidth requirements by limiting data access to locally relevant information.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent extracts and processes only the essential hypotheses needed for each pixel calculation, rather than accessing and processing all possible 3D information. By extracting and propagating only the most relevant hypotheses from previous iterations or neighboring pixels, the system reduces the volume of data that needs to be transferred across memory bandwidth, enabling efficient parallel processing on GPUs.

Inventive Principle:
Principle #2Taking out (Extraction)

Data Source

PatentEP3465611B1Apparatus and method for performing 3D estimation based on locally determined 3D information hypotheses
Publication Date: 2023.04.05 FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV
  • EP3465611B1 patent drawingFigure 1a
  • EP3465611B1 patent drawingFigure 1b
  • EP3465611B1 patent drawingFigure 2a~2b

AI summary

According to embodiments, an apparatus for performing 3D estimation on the basis of pictures of at least two different views comprises a hypotheses provider, a similarity measure calculator, and a 3D information determiner. The hypotheses provider is configured for locally determining 3D information hypotheses for positions of a current picture of a first view of the at least two different views on the basis of a pre-estimate which associates a 3D information estimate to each position of a picture of the first view. The similarity measure calculator is configured for calculating, for each position of the current picture of the first view, a similarity measure for each of the 3D information hypotheses of the respective position by measuring a similarity between a region of the current picture of the first view at the respective position and a corresponding region of a second view of the at least two different views located at a position displaced relative to the respective position by a respective 3D information hypothesis. The 3D information determiner is configured to select, for each position of the first view, the 3D information hypothesis of highest similarity measure.