Artificial Reality Scene Creation From Source Images and 3D Object Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for generating artificial reality environments require manual creation and precise placement of virtual objects, which is time-consuming and often results in deficient real-to-virtual object correspondence, especially when recreating real-world locations.
Innovation Solution
A creation system that employs machine learning models to automatically generate virtual objects by analyzing source images, using object identifiers and depth estimation to create 3D models with textures, and applies user-selected styles, enabling rapid and accurate placement of virtual objects in an artificial reality environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual creation and placement of virtual objects is used, then precision of object placement can be achieved, but time consumption increases significantly
Solution Approach 1:
The system enables automatic self-service by using machine learning models to autonomously generate virtual objects from source images and automatically place them in the artificial reality environment based on depth estimation and object identification, eliminating the need for manual intervention while maintaining placement accuracy
Solution Approach 2:
The patent replaces the manual mechanical process of object placement with an automated computational system that uses machine learning models, depth estimation algorithms, and automatic object generation to achieve precise placement without human intervention
2Ease of operation
If manual creation of virtual objects is used, then control over object details can be achieved, but ease of operation decreases
Solution Approach 1:
The system performs self-service by automatically generating virtual objects from source images using machine learning models, eliminating the need for users to manually create objects or possess technical expertise in 3D modeling and environment design
Solution Approach 2:
The patent uses copying by generating virtual objects that replicate real-world objects from source images through automatic object generation techniques, allowing users to create accurate representations without manual modeling effort
3Productivity
If automatic object generation is used, then productivity increases, but manufacturing precision may decrease
Solution Approach 1:
The patent replaces manual object creation with automated machine learning-based generation that uses depth estimation and object identification algorithms to maintain high precision in virtual object correspondence while dramatically increasing creation speed
Solution Approach 2:
The system changes parameters by using depth estimation and object identification metrics to guide automatic object generation, ensuring that generated virtual objects maintain accurate correspondence with real-world references while enabling rapid automated creation
Data Source
AI summary
Methods and systems described herein are directed to creating an artificial reality environment having elements automatically created from source images. In response to a creation system receiving the source images, the system can employ a multi-layered comparative analysis to obtain virtual object representations of objects depicted in the source images. A first set of the virtual objects can be selected from a library by matching identifiers for the depicted objects with tags on virtual objects in the library. A second set of virtual objects can be objects for which no candidate first virtual objects was adequately matched in the library, prompting the creation of a virtual object by generating depth data and skinning a resulting 3D mesh based on the source images. Having determined the virtual objects, the system can compile them into the artificial reality environment according to relative locations determined from the source images.