Generative AI 3D Object Creation in AR Spaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) technologies are limited in generating three-dimensional (3D) virtual objects that do not exist in reality and cannot modify generated objects, restricting the expansion of AR services and user interaction.
Innovation Solution
An electronic device equipped with a camera, processor, and generative artificial intelligence (AI) model that uses multi-modality information such as spatial images, user inputs, and 2D guide images to generate and modify 3D virtual objects, allowing users to create and customize virtual objects based on real-world space characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-modeled 3D virtual objects are used in AR technology, then the display of virtual objects in real-world space is enabled, but the ability to generate objects that do not exist in reality is limited
Solution Approach 1:
The patent replaces traditional mechanical 3D modeling methods with a deep neural network-based generative AI system. The neural network model learns to generate 3D virtual objects from 2D images and location data, substituting complex manual or algorithmic modeling processes with an intelligent system that can create novel objects that do not exist in reality.
Solution Approach 2:
The patent changes the fundamental parameters of object generation by transitioning from fixed pre-modeled objects to dynamically generated objects. The generative AI model adjusts parameters such as object shape, size, and characteristics based on learned patterns from training data, enabling creation of unique virtual objects rather than selecting from predefined options.
2Ease of manufacture
If deep neural network modeling is used to generate 3D virtual objects from 2D images, then object modeling capability is improved, but the ability to modify generated objects is lost
Solution Approach 1:
The patent introduces dynamic modification capabilities to the previously static generated objects. Users can interact with generated 3D virtual objects to modify their characteristics, such as changing size, shape, or other parameters. This transforms the system from a one-time generation process to an interactive, adaptable process where objects can be continuously adjusted after generation.
3Ease of operation
If traditional AR technologies are used, then virtual objects can be displayed in real-world space, but user interaction and customization options are restricted
Solution Approach 1:
The patent implements feedback mechanisms where user inputs during the object generation process influence the final generated object. The system accepts user inputs such as selected 2D images, location data, and modification preferences, processes these through the neural network, and generates corresponding 3D virtual objects. This feedback loop enables users to interact with and customize the generation process, significantly enhancing user control and AR service versatility.
Data Source
AI summary
An electronic device may include: a display; a camera configured to obtain an image; a memory storing at least one instruction; and at least one processor configured to execute the at least one instruction to: obtain spatial information about a real-world space based on the image obtained through the camera; obtain user inputs based on the image obtained through the camera; obtain object characteristic information from the user inputs; obtain object generation information for generating a virtual object, based on the spatial information and the object characteristic information; generate the virtual object for the object generation information by inputting the object generation information to a generative artificial intelligence (AI) model trained to generate a three-dimensional (3D) virtual object based on information about a space and an object; and control the display to display the virtual object.