Smart Glasses Assistant Suggestions Adapted to User Gaze
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users may not be aware of the full capabilities of automated assistants, leading to inefficient task handling and resource wastage when interacting with unfamiliar environments, as they manually engage with their devices without leveraging the assistant's functionality.
Innovation Solution
An automated assistant integrated with computerized glasses provides context-aware suggestions based on user gaze and input, limiting suggestions to relevant options, conserving resources and enhancing user experience by dynamically adapting to changes in context and visual scope.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the automated assistant provides a large number of suggestions based on all objects and locations in the visual scope, then the user can access more information and capabilities, but the computational resources and battery capacity of the computerized glasses are consumed more quickly
Solution Approach 1:
The system applies local quality by providing different levels of suggestion detail based on the user's current gaze focus. When the user gazes at a specific object or location, the assistant provides detailed suggestions related to that focal point, while providing fewer or more summarized suggestions for other areas in the visual scope. This selective information delivery reduces overall computational processing and battery consumption while maintaining information completeness for relevant items.
2Loss of information
If the automated assistant displays multiple suggestions simultaneously on the computerized glasses interface, then the user can see more options, but the display interface becomes cluttered and may obstruct the user's view
Solution Approach 1:
The system segments the display of suggestions by spatial location and relevance. Suggestions are divided into different categories (e.g., currently gazed-at objects vs. other objects in visual scope) and displayed in separate regions or with different levels of prominence. This segmentation allows multiple suggestions to be presented without creating a cluttered interface, as each suggestion group occupies its own organized space and can be accessed through gaze-based selection.
3Adaptability or versatility
If the automated assistant provides detailed suggestions for all objects in the environment, then the user can discover more capabilities, but the user experience becomes overwhelming and less efficient
Solution Approach 1:
The system implements dynamics by continuously adapting the set of displayed suggestions based on the user's real-time gaze direction and interaction history. As the user moves their gaze across different objects and locations, the assistant dynamically updates which suggestions are presented and with what level of detail. This dynamic adaptation ensures that the most relevant suggestions are always prominent, making the interface easy to operate while still allowing users to discover the full range of assistant capabilities through natural exploration.
Data Source
AI summary
Implementations set forth herein relate to an automated assistant that can render selectable suggestion(s) at a display interface of computerized glasses, and can adapt the suggestions according to changes to a gaze direction of the user and/or other further inputs from the user. The selectable suggestion(s) can be initially rendered based on contextual data that may be associated with a user who is directing their gaze into an environment that includes different environmental features. Certain environmental features can be identified by the automated assistant as being predicted to be of interest to the user and—when a user expresses interest in a particular feature—the selectable suggestions can be adapted. Interest of the user in the particular environmental feature can be expressed by redirecting their gaze towards the particular feature and/or providing further input relevant to the particular feature.