Voxel Occupancy Model Creation via Neural Network Projections
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for creating virtual 3D models from 2D images are either resource-intensive and prone to errors due to erroneous pixel correspondences, or they struggle with generalization when modeling unknown objects, particularly in augmented reality and robotics applications.
Innovation Solution
A method and system that utilize a neural network to determine voxel occupancy by analyzing projections of voxels across multiple images, employing projective pooling and recurrent neural networks to generate a voxel occupancy model, which represents the 3D structure of an object without requiring depth sensors and is less resource-intensive.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If geometric methods with exhaustive search along epipolar lines are used to determine pixel correspondences, then depth information can be computed, but the system is subject to erroneous pixel correspondences resulting in incorrect depth computations and consumes significant computational resources
Solution Approach 1:
The patent replaces the mechanical exhaustive search process along epipolar lines with a neural network-based system. The neural network directly predicts pixel correspondences and depth values from image data, eliminating the need for iterative geometric computations and reducing computational complexity while maintaining accuracy.
Solution Approach 2:
The patent introduces a neural network as an intermediary between image input and depth output. This intermediary learns the complex mapping from 2D image pixels to 3D depth values through training, serving as a mediator that avoids the need for explicit geometric computations and correspondence searching.
2Reliability
If multiple 2.5D depth images are fused into a single volumetric model using voxel grid rules, then a complete 3D model can be computed, but the process is time-consuming and consumes computer resources
Solution Approach 1:
The patent segments the 3D modeling task by processing each voxel independently through a neural network. Instead of fusing multiple depth images through iterative voxel grid operations, the system evaluates each voxel's occupancy probability separately based on projected image data, significantly reducing computation time.
Solution Approach 2:
The patent performs preliminary actions by pre-processing image data into feature representations and pre-computing voxel projections before the actual occupancy determination. This preparation allows the neural network to quickly evaluate voxel occupancy without performing time-consuming fusion operations during model creation.
3Adaptability or versatility
If deep learning methods compress 2D images into 1D feature vectors and use LSTM networks to generate 3D models, then the approach is suitable for imagining missing parts, but it leads to generalisation problems when modelling new unknown objects
Solution Approach 1:
The patent applies local quality by processing each voxel independently with its own neural network evaluation based on projected image data. This localized approach allows the system to adapt to local variations in unknown objects while maintaining overall model accuracy, improving generalization compared to global LSTM-based methods.
Solution Approach 2:
The patent changes the fundamental parameters of the deep learning approach by using a different network architecture (convolutional neural networks instead of LSTMs) and a different processing paradigm (voxel-wise evaluation instead of sequential frame processing). These parameter changes enable better generalization to unknown objects while maintaining the ability to handle incomplete data.
Data Source
AI summary
There is provided a method for creating a voxel occupancy model. The voxel occupancy model is representative of a region of space which can be described using a three-dimensional voxel array. The region of space contains at least part of an object. The method comprises receiving first image data, the first image data being representative of a first view of the at least part of an object and comprising first image location data, and receiving second image data, the second image data being representative of a second view of the at least part of an object and comprising second image location data. The method also comprises determining a first descriptor, the first descriptor describing a property of a projection of a first voxel of the voxel array in the first image data, and determining a second descriptor, the second descriptor describing a property of a projection of the first voxel in the second image data. The method also comprises assigning an occupancy value to the first voxel based on the first and second descriptors, the occupancy value being representative of whether the first voxel is occupied by the at least part of an object.