External AR Controller for Easier Smart Glasses Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing smart glasses platforms face challenges in intuitive interaction with virtual content due to steep learning curves for touch or gesture recognition, leading to resource wastage and reduced user experience.
Innovation Solution
An augmented reality system that uses a familiar external controller, such as a smartphone, to interact with virtual content by detecting inputs and modifying displayed objects, reducing the need for complex image processing on the AR device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If touch or gesture recognition is used for interaction with virtual content, then the AR device can detect user inputs, but the learning curve becomes steep and operation becomes difficult
Solution Approach 1:
The patent introduces an external controller as an intermediary device between the user and the AR system. The external controller captures user inputs (touch, gesture, voice) and transmits them to the AR device, simplifying the interaction model. Users interact with familiar external devices rather than learning complex AR-specific gestures, thereby improving ease of operation while reducing the perceived complexity of the AR system.
2Adaptability or versatility
If complex image processing is performed on the AR device to detect hand gestures, then gesture recognition is achieved, but hardware resources are wasted and energy is consumed
Solution Approach 1:
The patent extracts the complex image processing function from the AR device and relocates it to an external controller. The external controller performs the computationally intensive tasks of capturing images, detecting hand gestures, and recognizing gestures using its own processing resources. The AR device only receives the processed gesture recognition results, thereby conserving its limited hardware resources and reducing energy consumption while maintaining full gesture recognition capability.
3Measurement precision
If multiple machine learning models are used for hand gesture detection, then accurate gesture recognition is achieved, but the learning curve becomes steep and operation becomes difficult
Solution Approach 1:
The external controller serves as an intermediary that handles the complexity of multiple machine learning models internally. Users interact with the external controller using natural, familiar gestures without needing to understand or learn about the underlying multiple machine learning models. The external controller processes these gestures through its machine learning models and transmits the interpreted commands to the AR device, thereby maintaining high gesture detection accuracy while preserving ease of operation.
Data Source
AI summary
Systems and methods are provided for using an external controller with an AR device. The system establishes, by one or more processors of the AR device, a communication with an external client device. The system overlays, by the AR device, a first AR object on a real-world environment being viewed using the AR device. The system receives interaction data from the external client device representing one or more inputs received by the external client device and, in response, modifies the first AR object by the AR device.