Hybrid 3D Reconstruction Combining Stereo and Learning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current three-dimensional imaging methods, such as stereoscopic and learning-based reconstruction, face limitations in scene adaptability and precision, particularly with scenes lacking texture or experiencing occlusions, and require multiple images or prior learning data, respectively.

Innovation Solution

A dynamic three-dimensional imaging method that combines stereoscopic and learning-based reconstruction methods by generating intermediate models from two-dimensional images, selecting the best model portions based on quality criteria, and using the combined model for training, allowing for iterative refinement and improved performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If stereoscopic reconstruction method is used, then adaptability to various scenes is improved, but precision is degraded when areas lack texture or have reflections/occlusions

Engineering Contradiction:

Improveadaptability to various scenesVSAvoidreconstruction precision

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent combines stereoscopic reconstruction method and learning-based reconstruction method into a hybrid approach. The stereoscopic method provides adaptability to various scenes while the learning-based method compensates for precision losses in texture-less or occluded areas through neural network predictions, achieving both broad adaptability and high precision simultaneously.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces an intermediary mechanism where the learning-based reconstruction method acts as a complement to the stereoscopic method. When the stereoscopic method encounters difficulties (texture-less areas, reflections, occlusions), the learning-based method provides supplementary information through neural network predictions, mediating the overall reconstruction quality.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If learning-based reconstruction method is used, then adaptability to any scene type is improved, but precision is degraded compared to stereoscopic method

Engineering Contradiction:

Improveapplicability to any scene typeVSAvoidthree-dimensional model precision

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent merges learning-based reconstruction method with stereoscopic reconstruction method. The learning-based method provides broad applicability to any scene type through neural network training, while the stereoscopic method ensures high precision through traditional geometric reconstruction algorithms, achieving both universal applicability and high precision.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent segments the reconstruction process into two independent parts: one handled by the learning-based method for general applicability and another by the stereoscopic method for precision. Each method processes specific portions of the scene, and their results are combined, allowing each to optimize for its strength without compromising the other.

Inventive Principle:

Principle #1Segmentation

3Ease of manufacture

If stereoscopic reconstruction method is used, then no prior information is required, but multiple separate images are mandatory

Engineering Contradiction:

Improveno prior information requiredVSAvoidrequirement for multiple separate images

Core Design Contradiction:

Ease of manufactureVSDevice complexity

Solution Approach 1:

The patent combines the advantages of both methods: the stereoscopic method provides ease of use without prior information requirements, while the learning-based method reduces the need for multiple separate images by using neural network predictions from single or few images. The combination achieves both ease of operation and reduced image quantity requirements.

Inventive Principle:

Principle #5Merging (Combining)

4Device complexity

If learning-based reconstruction method is used, then single image processing is possible, but prior learning data is mandatory

Engineering Contradiction:

Improvesingle image processing capabilityVSAvoidrequirement for prior learning data

Core Design Contradiction:

Device complexityVSEase of manufacture

Solution Approach 1:

The patent merges learning-based method (which enables single image processing) with stereoscopic method (which requires no prior information). The learning-based component handles single image input through neural networks, while the stereoscopic component provides geometric constraints without requiring additional prior learning data, achieving both single image capability and ease of use.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11908070B2Dynamic three-dimensional imaging method

Publication Date: 2024.02.20 QUERBES OLIVIER

AI summary

Disclosed is a dynamic three-dimensional imaging method that allows the generation of a three-dimensional numerical model that represents an observed three-dimensional scene. The generated model optimally combines (404) data from two intermediate three-dimensional numerical models, respectively obtained by a stereoscopic three-dimensional reconstruction calculation method (402) and by a method of three-dimensional reconstruction calculation through learning (403). In addition, each new model that is generated helps to improve the overall performance of the method of three-dimensional reconstruction calculation by learning.