External AR Controller for Low-Learning-Curve Smart Glasses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing smart glasses platforms face challenges in intuitive interaction with virtual content due to steep learning curves for touch or gesture recognition, leading to inefficient use of hardware resources and user frustration.
Innovation Solution
An AR system that allows interaction with virtual content using a familiar external controller, such as a smartphone, which detects inputs and modifies displayed content, reducing the need for complex image processing on the AR device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If touch or gesture recognition is implemented in smart glasses for interacting with virtual content, then interaction capability is improved, but the learning curve increases and ease of operation deteriorates
Solution Approach 1:
The patent introduces an external computing device (smartphone, tablet, or computer) as an intermediary that handles the complex touch and gesture recognition processing. The smart glasses only need to detect basic user inputs and communicate with the external device, which then processes the interactions and sends commands back to the glasses. This mediator approach allows advanced interaction capabilities without requiring the glasses themselves to have sophisticated processing built-in, thereby reducing the learning curve for users.
Solution Approach 2:
The patent extracts the complex image processing and gesture recognition functions from the smart glasses and relocates them to an external computing device. By taking out these resource-intensive operations from the limited hardware of the glasses, the system achieves sophisticated interaction capabilities while the glasses remain simple to operate. The external device handles the computational burden, allowing the glasses to focus on display and basic input detection.
2Measurement precision
If complex image processing is performed on the AR device for gesture recognition, then interaction accuracy is improved, but hardware resource consumption increases
Solution Approach 1:
The external computing device serves as an intermediary that performs the computationally intensive image processing and gesture recognition algorithms. The smart glasses capture images and transmit them to the external device, which then processes the gestures with high accuracy using its more powerful hardware. This division of labor allows accurate gesture detection without the smart glasses needing to consume excessive energy for processing.
Solution Approach 2:
The patent utilizes the external computing device as a copy of the processing capability that would otherwise need to exist in the smart glasses. Instead of duplicating full image processing power in the glasses, the system creates a communication link to an external device that can perform these operations. This copying approach allows the glasses to leverage the processing power of the external device without having to physically contain that processing capability, thereby conserving energy and hardware resources.
3Adaptability or versatility
If embedded sensors are used for touch input detection in smart glasses, then interaction functionality is improved, but device complexity increases
Solution Approach 1:
The patent makes the external computing device universal by allowing it to handle multiple interaction modalities (touch, gesture, voice) and serve as the central processing unit for all interaction types. The smart glasses can use simple embedded sensors for basic input detection, while the external device provides universal processing for all interaction functionalities. This multi-functionality in the external device reduces the need for specialized components in the glasses, thereby reducing overall system complexity.
4Adaptability or versatility
If hand gesture recognition with machine learning models is implemented, then interaction versatility is improved, but processing time and energy consumption increase
Solution Approach 1:
The external computing device acts as an intermediary that performs the time-consuming machine learning-based gesture recognition. The smart glasses capture images and send them to the external device, which then runs the machine learning models to recognize gestures. This intermediary approach allows sophisticated gesture recognition with multiple machine learning models without the processing time being a bottleneck in the glasses, as the external device has more powerful processing capabilities.
Solution Approach 2:
The system performs preliminary action by pre-processing images and preparing them for gesture recognition on the external device before transmitting them back to the glasses for display. The external device can perform preliminary analysis and filtering of gestures, reducing the overall processing time required. By doing the heavy lifting of machine learning inference in advance on a more powerful device, the system reduces the time loss that would occur if all processing had to happen in real-time on the limited hardware of the smart glasses.
Data Source
AI summary
Systems and methods are provided for using an external controller with an AR device. The system establishes, by one or more processors of the AR device, a communication with an external client device. The system overlays, by the AR device, a first AR object on a real-world environment being viewed using the AR device. The system receives interaction data from the external client device representing one or more inputs received by the external client device and, in response, modifies the first AR object by the AR device.