3D AR Object Generation from Real-World Scenes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) applications lack the ability to select and incorporate real-world objects into three-dimensional AR scenes, relying instead on pre-stored information and models.
Innovation Solution
A method and system for capturing and selecting real-world objects using a media acquisition unit, performing coarse semantic segmentation, deriving connected contours, generating intermediate contours, and configuring the objects with a three-dimensional mesh and texture mapping to create a realistic 3D AR model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-stored real-world information is used to generate AR models, then the AR model generation process is simple and fast, but the versatility and realism of the AR models are limited
Solution Approach 1:
The system creates a digital copy of the real-world object by capturing its image and generating a 3D model from that capture. This allows the physical object to be replicated virtually without needing to physically manipulate or store the original object, thereby achieving versatility while managing complexity through digital representation
Solution Approach 2:
The system introduces an intermediary processing pipeline between the real-world object and the final AR model. This intermediary layer includes image capture, object detection, 3D model generation, and texture mapping components that bridge the gap between physical and digital domains, enabling versatile AR model creation from real-world objects
2Manufacturing precision
If real-world objects are captured and converted to 3D AR models, then the realism and versatility of AR applications are enhanced, but the processing time and computational complexity increase
Solution Approach 1:
The system performs preliminary actions by capturing the real-world object and generating a base 3D model before final AR integration. This preliminary processing separates the complex tasks into stages: capture, 3D generation, texture mapping, and AR scene integration, allowing optimization at each stage and reducing overall processing time
Solution Approach 2:
The complex process of converting a real-world object to a 3D AR model is segmented into distinct operational steps: image capture, object detection and segmentation, 3D model generation, texture mapping, and AR scene composition. This segmentation allows each step to be optimized independently and improves overall efficiency
3Ease of operation
If users can select and capture any real-world object, then the ease of operation and creativity are improved, but the difficulty of detecting and measuring objects increases
Solution Approach 1:
The system provides self-service functionality by automatically detecting objects, generating 3D models, and preparing AR scenes without requiring manual user intervention for each step. Users simply point the camera at an object and press a capture button, while the system handles the complex detection and measurement processes automatically
Solution Approach 2:
The system incorporates feedback mechanisms where the detected object information is immediately fed back into the 3D model generation process. This feedback loop allows the system to adjust its detection and measurement parameters based on the captured object's characteristics, improving accuracy while maintaining ease of operation
Data Source
AI summary
A method and system for generating three-dimensional (3D) model augmented related objects from a scene are provided. The method includes creating one or more 3D objects and placing the 3D objects into the 3D scene. Embodiments herein disclose methods and systems for generating 3D augmented reality (AR) objects from a scene. The method may capture an object from the scene, perform a coarse semantic segmentation on the identified object, derive connected contour, generate intermediate contour from at least one outer and inner contours, and configure three-dimensional mesh and texture mapping to generate a three-dimensional model of the captured object.