3D Reconstruction via Complementary View Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D reconstruction methods based on 2D images with limited views often result in multi-isotropic reconstructions due to occlusion and other issues, leading to unsatisfactory reconstruction quality.
Innovation Solution
A method that involves extracting original image features from a 2D image, determining an original 3D object, calculating a camera pose for a complementary view, generating a complementary 2D image, performing 3D reconstruction, and fusing the original and complementary 3D objects to enhance reconstruction quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If more camera views are selected for photographing the target object, then the reproduction quality of the 3D object is improved, but the difficulty of obtaining complete 2D images increases due to occlusion and geographic location constraints
Solution Approach 1:
The patent applies preliminary action by predicting complementary 2D images from the original 2D image before actual photographing. The system uses a prediction model to generate images from viewpoints that would be occluded or difficult to capture, allowing the 3D reconstruction to proceed as if complete multi-view images were available. This resolves the contradiction by preparing the needed information in advance through computational prediction rather than physical capture.
Solution Approach 2:
The patent uses copying by generating synthetic complementary 2D images that replicate what would be captured from occluded viewpoints. Instead of physically capturing images from all desired angles, the system creates copies of the visual information through image synthesis and rendering based on the original image and predicted camera poses. This allows the 3D reconstruction to utilize information from viewpoints that were never actually captured.
2Ease of operation
If 2D images with limited views are used for 3D reconstruction, then the ease of operation is improved, but the reconstruction results become multi-isotropic and unsatisfactory
Solution Approach 1:
The patent introduces an intermediary element - the predicted complementary 2D images - that bridges the gap between limited captured views and complete multi-view reconstruction. These intermediary images serve as a mediator that provides the missing visual information needed for high-quality 3D reconstruction without requiring actual multi-view photography. The system uses these intermediate synthetic images to enhance the limited input data into sufficient reconstruction input.
Solution Approach 2:
The patent applies dimensionality change by transitioning from working solely with captured 2D image dimensions to incorporating predicted 2D images from additional viewpoint dimensions. By synthesizing images from occluded viewpoints that represent different spatial dimensions, the system enriches the input data beyond the limitations of the original single or few captured views, enabling comprehensive 3D reconstruction from limited input.
Data Source
AI summary
A method, device, system and computer readable storage medium for 3D reconstruction are provided. The method includes: extracting original image features from an original 2D image of a target object; determining an original 3D object based on the original image features; determining a camera pose in the complementary view of the target object, where the complementary view is different from a first view in which the original 2D image is generated; generating a complementary 2D image of the target object in the complementary view based on the camera pose in the complementary view; performing 3D reconstruction based on the complementary 2D image to generate a complementary 3D object corresponding to the complementary 2D image; fusing the original 3D object and the complementary 3D object to obtain a 3D reconstruction result of the target object.