Interactive 3D Reconstruction With Local Resampling Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D reconstruction methods, particularly deep learning-based implicit reconstruction systems, struggle to accurately reproduce complex details and shapes of objects not encountered in the training set, often over-smoothing or missing parts in the reconstruction process.
Innovation Solution
A hybrid approach combining a global encoder-decoder network for initial reconstruction and a local feature encoder-decoder network for targeted resampling and refinement, allowing iterative user interaction to enhance detail accuracy by merging global and local reconstructions based on user-defined points of interest.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple images are captured from different positions and angles to improve 3D reconstruction accuracy, then the completeness of the target surface is improved, but the number of images required increases significantly, making the process time-consuming and laborious
Solution Approach 1:
The system performs preliminary action by automatically capturing images from multiple angles and positions before the user manually positions the device. The image capturing unit automatically moves to different positions and captures images in advance, eliminating the need for users to manually position and capture each image separately.
Solution Approach 2:
The system implements self-service through automatic image capture functionality where the image capturing unit autonomously positions itself and captures images from multiple angles without requiring manual intervention. The control unit automatically processes these images to generate 3D data, making the system self-sufficient in the image collection process.
2Speed
If images are captured in advance to reduce on-demand processing time, then response speed is improved, but stored image data may be incomplete or inaccurate if the target moves or changes
Solution Approach 1:
The system applies dynamics by enabling real-time image capture and 3D data generation. When a measurement request is received, the image capturing unit immediately captures images from multiple angles and the control unit generates updated 3D data in real-time, ensuring the data reflects the current state of the target rather than relying on potentially outdated pre-captured images.
3Measurement precision
If manual positioning and image capture is performed to ensure image quality, then measurement accuracy is improved, but the complexity of operation increases and requires skilled operators
Solution Approach 1:
The system performs self-service by automatically capturing images from multiple angles and positions without requiring manual intervention. The image capturing unit autonomously positions itself and captures images, while the control unit automatically processes these images to generate 3D data, eliminating the need for skilled operators to manually position and capture images.
Solution Approach 2:
The system replaces the mechanical manual positioning and image capture process with an automated image capturing unit controlled by a control unit. This substitution of mechanical manual operations with an automated control system simplifies the user interface and reduces operational complexity while maintaining or improving image quality through consistent automated positioning.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for 3D reconstruction of a target is provided, comprising obtaining an initial global reconstruction of the target in a 3D space, inferred by a global machine learning model; providing, to a user, an initial visualisation of the target based on the reconstruction; receiving, from the user, at least one indication of at least one point of interest in the visualisation; resampling at least one first subsection of the target based on the at least one point of interest to obtain local data, wherein the local data is associated with the subsection based on spatial information that associates the local data with a point in 3D space; inputting the resampled local data and spatial information into a local feature machine learning model to obtain at least one 3D reconstruction of the target, wherein the local feature machine learning model has been trained to output a target reconstruction from local data of resampled subsections, and wherein the 3D coordinate system of the local 3D reconstruction aligns with the global 3D reconstruction; and merging the global 3D reconstruction with the local 3D reconstruction. A corresponding computer system and computer readable medium may also be provided.