3D Reconstruction Using Local-Global Neural Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing high-precision three-dimensional reconstruction technologies are limited by high costs and complexity, making them unsuitable for widespread adoption in virtual reality applications due to the need for large sensor arrays and significant computing power, and suffer from reconstruction inaccuracies due to sensor measurement errors and image data quality issues.
Innovation Solution
A method combining local pose optimization, neural network prediction, global pose optimization, and neural network completion to achieve high-quality and high-precision three-dimensional reconstruction, utilizing a hierarchical reconstruction strategy that reduces the need for extensive sensor arrays and computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If high-precision three-dimensional reconstruction is achieved using complex sensor devices and computing devices, then manufacturing precision and measurement precision are improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent divides the three-dimensional reconstruction process into multiple stages: initial model construction from sparse views, iterative refinement stages, and final optimization. Each stage processes a subset of images and progressively improves the model, avoiding the need to process all images simultaneously as in traditional methods.
Solution Approach 2:
The patent performs preliminary construction of a three-dimensional model using only a subset of images before all images are fully processed. This initial model serves as a foundation that is subsequently refined through iterative optimization, enabling early visualization and reducing overall processing complexity.
2Measurement precision
If a huge sensor array composed of many cameras is used to achieve high-precision three-dimensional reconstruction, then measurement precision is improved, but device complexity and cost increase significantly
Solution Approach 1:
The patent uses a partial action approach by processing a subset of images in each iteration rather than requiring all images to be processed simultaneously. The method performs multiple passes through the image set, progressively refining the model with different subsets, which reduces the effective number of cameras needed at any given time.
3Manufacturing precision
If huge computing power is used to provide computing support for the reconstruction process, then manufacturing precision is improved, but use of energy and cost increase
Solution Approach 1:
The patent segments the computational workload into multiple iterative stages, where each stage processes a portion of the data and produces an intermediate result. This divides the huge computing task into manageable chunks that can be performed sequentially with moderate computing resources rather than requiring all resources simultaneously.
Solution Approach 2:
The patent performs preliminary processing and feature extraction on images before the main reconstruction computation. By pre-processing images to extract key features and reduce data dimensionality, the method reduces the computational burden of the subsequent three-dimensional model construction and optimization stages.
4Device complexity
If three-dimensional reconstruction is performed using consumer-grade sensors, then device complexity and cost are reduced, but measurement precision deteriorates due to sensor measurement errors and image data quality issues
Solution Approach 1:
The patent implements feedback mechanisms where the three-dimensional model is continuously refined based on comparison with input images. The optimization process uses photometric error and geometric constraints as feedback to adjust model parameters, compensating for sensor inaccuracies and improving precision even with consumer-grade sensors.
Solution Approach 2:
The patent replaces reliance on high-precision mechanical sensor systems with computational methods. Instead of depending on precise sensor measurements, the method uses algorithmic optimization, neural networks, and iterative refinement to achieve accurate three-dimensional reconstruction from lower-quality input data.
Data Source
AI summary
A three-dimensional reconstruction method, a system, and a non-transitory computer readable storage medium are disclosed herein. The method includes: performing local pose optimization by using a target image frame to obtain a local pose error; performing neural network prediction on the target image frame to obtain an initial reconstruction error; performing three-dimensional reconstruction according to the local pose error and the initial reconstruction error to obtain an initial reconstruction model; performing global pose optimization by using historical image frames to obtain a global optimization result and a global pose error; performing neural network completion on the global optimization result to obtain a final reconstruction error; and optimizing the initial reconstruction model according to the global pose error and the final reconstruction error to obtain a final reconstruction model. The three-dimensional reconstruction method can achieve high-quality and high-precision three-dimensional reconstruction more quickly and conveniently.