Smart Glasses 3D AR via 2D Image Depth Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality technologies, particularly in smart glasses, are limited by two-dimensional (2-D) recognition, which fails to accurately represent spatial coordinates in three-dimensional (3-D) environments, leading to a mismatch between virtual and real-world objects, making it difficult to implement effective holographic displays in industrial settings.
Innovation Solution
A method that collects two-dimensional (2-D) image data from smart glasses, generates 2-D coordinate data, and uses location, rotation, and depth information to calculate three-dimensional (3-D) coordinates, allowing for the accurate display of augmented objects in 3-D space on holographic displays, without requiring costly 3-D data gathering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If 2-D recognition technology is used in smart glasses, then device complexity is reduced and ease of manufacture is improved, but measurement precision of spatial coordinates deteriorates and reliability of augmented reality implementation deteriorates
Solution Approach 1:
The patent transforms 2-D image coordinates into 3-D spatial coordinates by introducing depth estimation through machine learning models. This dimensionality change allows the system to maintain simplicity in data collection (using only 2-D cameras) while achieving accurate 3-D spatial representation for augmented reality applications.
Solution Approach 2:
The patent introduces machine learning models as an intermediary component that processes 2-D image data and converts it into accurate 3-D spatial coordinates. This intermediary layer bridges the gap between simple 2-D data collection and the need for precise 3-D spatial understanding, resolving the contradiction between ease of manufacture and measurement precision.
2Ease of operation
If 2-D recognition technology is used in smart glasses, then device complexity is reduced and ease of operation is improved, but reliability of augmented reality implementation deteriorates
Solution Approach 1:
By converting 2-D coordinates to 3-D coordinates through depth estimation, the system maintains ease of operation with simple camera-based input while achieving reliable augmented reality placement that accurately represents real-world spatial relationships.
Solution Approach 2:
Machine learning models serve as an intermediary that enhances reliability by accurately inferring 3-D spatial information from 2-D images, enabling reliable augmented reality object placement without complicating the operation or requiring additional sensors.
3Measurement precision
If 3-D data collection is implemented, then measurement precision of spatial coordinates is improved and reliability of augmented reality is improved, but device complexity increases and manufacturing cost increases
Solution Approach 1:
Instead of collecting 3-D data directly through complex sensors, the patent inverts the approach by collecting simple 2-D image data and computationally reconstructing 3-D spatial coordinates through machine learning models. This inversion achieves high measurement precision while maintaining low device complexity.
Solution Approach 2:
The patent replaces mechanical 3-D sensing systems with a computational approach using machine learning models that process 2-D image data. This substitution eliminates the need for complex hardware while achieving accurate 3-D spatial measurement through software-based depth estimation.
Data Source
AI summary
Approaches presented herein enable implementation of augmented reality in a smart glasses device. More specifically, two-dimensional (2-D) image data of a real-world object is collected from a 2-D camera of the smart glasses device. From the collected 2-D image data, 2-D coordinate data is generated. Based on location and rotation data of the smart glasses device and 2-D depth information from a viewing angle of the smart glasses device, three-dimensional (3-D) coordinates are generated from the generated 2-D coordinate data. An augmented object is displayed, on a holographic display of the smart glasses device, at an apparent location of the real-world object utilizing the 3-D coordinates.