See-through Display Gaze Recognition External Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
See-through display systems face limitations in analyzing and reacting to objects in various real-world backgrounds and determining the user's focus, which restricts their ability to provide enhanced experiences across different environments.
Innovation Solution
A see-through display system equipped with gaze detection and outward-facing image sensors acquires and utilizes information about objects in the user's field of view, allowing for the identification of objects on external displays and enabling contextual information retrieval and interaction through gaze-based inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If see-through display systems use basic display functionality without advanced sensing, then device complexity is reduced, but the ability to analyze and react to objects in real-world backgrounds and determine user focus is limited
Solution Approach 1:
The patent combines multiple sensing capabilities (outward-facing image sensor, gaze detection subsystem, depth sensor) with the see-through display system to create an integrated system that can detect and analyze external objects and user focus, thereby resolving the contradiction between enhanced adaptability and increased complexity by merging functions into a unified system architecture
Solution Approach 2:
The see-through display system is designed to perform multiple functions including displaying virtual imagery, capturing external scenes, detecting user gaze, identifying objects, and providing contextual information, making the system universal and adaptable to various interactions while managing complexity through integrated multi-functionality
2Ease of operation
If see-through display systems incorporate gaze detection and image sensors to identify objects, then user interaction capability is enhanced, but device complexity increases
Solution Approach 1:
The system automatically performs object identification and contextual information retrieval based on detected user gaze without requiring manual input from the user, enabling seamless interaction while managing complexity through automated processing of sensing data and image analysis
Solution Approach 2:
The system provides real-time feedback to the user by displaying contextual information about detected objects based on gaze detection, creating an interactive loop that enhances ease of operation while the feedback mechanism itself manages system complexity through structured information flow
3Speed
If the system processes and analyzes images from external displays in real-time, then responsiveness to user gaze is improved, but computational resource consumption increases
Solution Approach 1:
The system pre-loads and caches contextual information for detected objects, and uses efficient image processing techniques to quickly match detected objects with stored information, enabling rapid response to user gaze while reducing real-time computational burden and energy consumption
Solution Approach 2:
The system processes only the specific region of interest where user gaze is directed rather than analyzing the entire external scene, and retrieves only relevant contextual information needed for the current interaction, reducing computational resource consumption while maintaining responsiveness
Data Source
AI summary
Embodiments are disclosed that relate to the recognition via a see-through display system of an object displayed on an external display device at which a user of the see-through display system is gazing. For example, one embodiment provides a method of operating a see-through display system comprising acquiring an image of an external display screen located in the background scene via an outward facing image sensor, determining via a gaze detection subsystem a location on the external display screen at which the user is gazing, obtaining an identity of an object displayed on the external display screen at the location determined, and performing an action based upon the identity of the object.