Virtual Depth Map Generation via Neural Network Image Concatenation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for three-dimensional (3D) model reconstruction in industrial applications are economically inefficient due to the high cost of dedicated depth acquisition devices, necessitating a low-cost technique for obtaining depth information.

Innovation Solution

A method involving the combination of multiple two-dimensional images to generate an intermediate image, processed by a neural network to produce a depth map and confidence map, allowing for the extraction of a corresponding depth map for 3D model reconstruction without the need for expensive depth sensors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If dedicated depth acquisition devices (structured light, ToF) are used to capture depth information, then measurement precision of depth is improved, but device cost increases significantly

Engineering Contradiction:

Improvedepth information accuracyVSAvoiddevice cost

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The patent creates virtual depth information by copying and processing 2D image data through neural networks, rather than using physical depth sensors. Multiple 2D images are processed to generate depth maps, effectively creating a software-based copy of depth acquisition functionality that avoids expensive hardware

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces mechanical/optical depth sensing systems (structured light projectors, ToF sensors) with a computational approach using neural networks and image processing algorithms. This substitution eliminates the need for complex physical depth acquisition devices while maintaining depth measurement capability

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Ease of manufacture

If multiple 2D images are combined and processed by neural networks to generate depth maps, then device cost is reduced, but measurement precision of depth information deteriorates

Engineering Contradiction:

Improvedevice costVSAvoiddepth information accuracy

Core Design Contradiction:

Ease of manufactureVSMeasurement precision

Solution Approach 1:

The patent performs preliminary actions by collecting and preprocessing multiple 2D images before depth calculation. Images are captured from different positions and angles, then concatenated and processed by the neural network to generate accurate depth maps, ensuring quality input data for depth estimation

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements feedback mechanisms through confidence maps that indicate the reliability of predicted depth values. The system uses loss functions during training to compare predicted depth maps with ground truth, continuously improving accuracy. Confidence maps provide feedback on which depth regions are reliable, allowing for quality control of the generated depth information

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11721006B2Method and device for generating virtual reality data

Publication Date: 2023.08.08 REALSEE (BEIJING) TECHNOLOGY CO LTD

AI summary

A method for generating an image of a scene with a corresponding depth map is disclosed herein. The method comprises collecting a plurality of copies of a two-dimensional image of the scene, generating an intermediate image by concatenating the plurality of copies of the two-dimensional image along a first direction, generating an intermediate depth map corresponding to the intermediate image by applying a neural network to the intermediate image, and generating, from the intermediate image and the intermediate depth map, the image of the scene with the corresponding depth map.