Neural Network Scene Modeling for Real-Time Depth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for creating a three-dimensional model of a physical scene from images are too processing-intensive for real-time visualization, and active registration techniques provide lower resolution, making them unsuitable for achieving realistic results.
Innovation Solution
The method employs an artificial neural network to match pixel values between images, using a database of image patterns and initial filtering to enhance efficiency and accuracy, allowing for real-time depth map generation and visualization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional image registration algorithms (e.g., Pollefeys' algorithm) are used to create depth maps from image sequences, then the modeling accuracy and realism are improved, but the processing time increases significantly making real-time visualization impossible
Solution Approach 1:
The patent divides the image processing task into multiple stages: initial filtering to identify candidate matches, neural network-based refinement to determine precise correspondences, and depth map generation. This segmentation allows each stage to focus on specific aspects of the problem, improving overall efficiency while maintaining accuracy.
Solution Approach 2:
The patent replaces traditional mechanical/image processing algorithms with an artificial neural network system. The neural network learns optimal matching strategies from training data and can perform depth estimation in real-time, substituting the computationally intensive polynomial registration algorithms with a more efficient learned model.
2Productivity
If active registration procedures (e.g., laser or infra red projection) are used to generate depth maps faster, then the processing time is reduced enabling faster generation, but the resolution is substantially lower preventing satisfying realism
Solution Approach 1:
The patent introduces an artificial neural network as an intermediary between the input images and the depth map output. The neural network processes the image data through learned representations, achieving both real-time processing speeds and high-resolution output quality, thereby mediating between the conflicting requirements of speed and precision.
3Measurement precision
If the entire second image is scanned to find matching pixels for every first image portion, then matching accuracy is improved, but the computational complexity and processing time increase significantly
Solution Approach 1:
The patent performs preliminary filtering of the second image to identify candidate matching regions before the neural network performs detailed matching. This preliminary action reduces the search space significantly, allowing the system to maintain high matching accuracy while reducing computational complexity by focusing resources on promising candidate regions only.
Data Source
AI summary
The present invention relates to automatic modeling of a physical scene. At least two images (I1, I2) of the scene are received, which are taken from different angles and/or positions. A matching module (130) matches image objects in the first image (I1) against image objects in the second image (I2), by first loading pixel values for at least one first portion of the first image (I1) into an artificial neural network (133). Then, the artificial neural network (133) scans the second image (I2) in search of pixels representing a respective second portion corresponding to each of the at least one first portion; determines a position of the respective second portion upon fulfillment of a match criterion; and produces a representative matching result (M12). Based on the matching result (M12), a first calculation module (140) calculates a fundamental matrix (F12), which defines a relationship between the first and second images (I1, I2). Based on the fundamental matrix (F12), in turn, a second calculation module (150) calculates a depth map (D12), which describes distance differences between a set of image points in the first image (I1) and a corresponding set of image points in the second image (I2). Finally, the depth map (D12) constitutes a basis for a synthetic model of the scene.