Electronic Eyewear AR Search Using Visual and Voice Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing wearable electronic devices struggle to provide contextually relevant augmented reality features based on user voice commands and visual inputs, often resulting in inappropriate or irrelevant search results.
Innovation Solution
An electronic eyewear device equipped with cameras and microphones processes visual and audio inputs to capture images and voice commands, using contextual information to refine search results and provide augmented reality features that match the user's intent and the viewed environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the electronic eyewear device processes both visual and voice inputs to provide contextually relevant augmented reality features, then the relevance and accuracy of search results improve, but the device complexity increases
Solution Approach 1:
The patent combines visual input processing (camera-based object recognition) and voice input processing (microphone-based command recognition) into a unified augmented reality search system. The processing circuit integrates both modalities to provide contextually relevant search results, resolving the contradiction by merging multiple data sources to improve accuracy while accepting the necessary complexity increase
Solution Approach 2:
The electronic eyewear device is designed with multi-functionality to handle both visual and voice inputs through a single integrated system. The processing circuit serves multiple functions: capturing visual data, capturing voice data, processing both types of data, and generating contextually relevant augmented reality features, thereby improving search accuracy through universal processing capabilities
2Measurement precision
If the device uses multiple sensors and processing circuits to analyze visual and audio context, then the contextual relevance of search results improves, but the power consumption increases
Solution Approach 1:
The system performs preliminary processing of visual and audio data to extract contextual information before generating final augmented reality features. By pre-processing sensor inputs to identify objects, scenes, and user intent in advance, the system improves contextual relevance while managing power consumption through efficient processing stages
3Adaptability or versatility
If the electronic eyewear device integrates direct communication with backend services and third-party APIs, then the functionality and versatility improve, but the device complexity increases
Solution Approach 1:
The processing circuit acts as an intermediary between the electronic eyewear device and external services including backend services and third-party APIs. It manages communications with text-to-speech services, the SHAZAM PLAYER app, and other external systems, enabling enhanced functionality while centralizing complexity management in a dedicated communication layer
Data Source
AI summary
Augmented reality features are selected for presentation to a display of an electronic eyewear device by using a camera of the electronic eyewear device to capture a scan image and processing the scan image to extract contextual signals. Simultaneously, voice data from the user is captured by a microphone of the electronic eyewear device and voice-to-text conversion of the captured voice data is performed to identify keywords in the voice data. The extracted contextual signals and the identified keywords are then used to select at least one augmented reality feature that matches the extracted contextual signals and the identified keywords, and the selected augmented reality feature is presented to the display for user selection. The contextual information thus refines the search results to provide the augmented reality feature best suited for the context of the scan image captured by the electronic eyewear device.