Near-Eye Display Input Control Using Context-Aware Event Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional near-eye displays struggle to accurately distinguish between user interactions with real-world tasks and virtual content tasks, leading to difficulties in identifying user input events such as hand gestures, eye movements, and voice commands.
Innovation Solution
A user input framework that combines sensor data from near-eye displays and paired devices with custom-trained UI contextual data using a transformer encoder-decoder to generate a context value, enhancing the accuracy of input event detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor data alone is used for user input detection, then device complexity is reduced, but measurement precision deteriorates due to inability to distinguish real-world tasks from virtual content interactions
Solution Approach 1:
The patent combines sensor data with UI contextual data from a transformer encoder-decoder model to generate an enhanced user input score. This merging of multiple data sources (sensor inputs and contextual patterns) enables the system to distinguish between real-world tasks and virtual content interactions more accurately, resolving the contradiction between detection accuracy and system complexity by integrating complementary information streams.
Solution Approach 2:
The transformer encoder-decoder model acts as an intermediary that processes both sensor data and historical interaction patterns to generate contextual representations. This intermediary component translates complex interaction patterns into usable contextual data that enhances input detection accuracy without requiring direct complex processing of all sensor data alone.
2Measurement precision
If sensor data processing is simplified, then processing time is reduced, but user input event identification accuracy deteriorates
Solution Approach 1:
The system performs preliminary processing by pre-computing contextual representations using the transformer encoder-decoder model on historical interaction data. This preliminary action creates ready-to-use contextual patterns that can be quickly matched against current sensor inputs, reducing real-time processing requirements while maintaining high identification accuracy through pre-processed contextual knowledge.
Solution Approach 2:
The patent replaces direct complex mechanical processing of sensor data with a computational model-based approach using the transformer encoder-decoder. This substitution uses AI/ML algorithms to process and interpret sensor data patterns, achieving higher accuracy without proportionally increasing processing time through optimized computational methods rather than brute-force analysis.
3Reliability
If contextual data integration is added, then user input accuracy is improved, but device complexity increases
Solution Approach 1:
The system incorporates feedback mechanisms where the transformer encoder-decoder model continuously learns from historical interaction patterns and refines its contextual representations. This feedback loop allows the system to improve reliability by adapting to user behavior patterns over time, using past performance data to enhance future input detection accuracy while managing complexity through iterative optimization rather than requiring overly complex static systems.
Data Source
AI summary
A near-eye display includes a processor to generate a sensor input value based on sensor data received from one or more sensors associated with the near-eye display and generate a context value based on a contextual score indicating a user state associated with the near-eye display. The contextual score is based in part on previous user interactions with a user interface of the near-eye display. The processor is also configured to compute an input event value based on the sensor input value and the context value and determine whether to trigger a change in virtual content displayed by the near-eye display based on comparing the input event value to a threshold.