Confidence-Based Touchless UX via Multi-Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current electronic devices, such as smart home devices, rely heavily on voice or touch inputs, which can be disruptive and compromise user experience by not considering their physical context, failing to provide contextual and proactive user interactions.
Innovation Solution
An electronic device configured to initiate touchless user experience operations based on proximity and gesture detection using multiple sensors, such as IR, ultrasound, and cameras, with confidence-based processing to adapt and provide relevant information and controls accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If voice or touch inputs are used to initiate functions, then user interaction capability is improved, but user experience is worsened due to disruption of regular activities
Solution Approach 1:
The system performs preliminary detection of presence and gesture using sensors (camera, ultrasonic sensor, IR sensor) before initiating any user interaction. This allows the device to prepare for interaction in advance, detecting when a user is approaching or present in the interaction zone, and only then activating the appropriate interface, thereby avoiding unnecessary disruptions to users who are not seeking interaction
Solution Approach 2:
The patent replaces traditional mechanical interaction methods (voice commands and touch inputs) with optical and acoustic field-based detection methods. Using camera imaging, ultrasonic wave propagation, and infrared radiation detection, the system can sense user presence and gestures without requiring physical or vocal input from the user, thus eliminating the disruptive nature of voice and touch inputs while maintaining interaction capability
2Measurement precision
If multiple sensors are used for proximity and gesture detection, then measurement precision is improved, but device complexity is worsened
Solution Approach 1:
The detection system is segmented into distinct functional modules: a camera module for visual gesture detection, an ultrasonic sensor module for proximity detection through sound wave analysis, and an IR sensor module for thermal presence detection. Each sensor type handles specific detection tasks, and their results are processed independently before being integrated through confidence-based fusion. This segmentation allows each component to be optimized for its specific function while maintaining overall system manageability
Solution Approach 2:
The patent introduces a confidence-based processing intermediary that acts as a mediator between multiple sensor inputs and the final gesture recognition output. Each sensor generates confidence scores for detected features, and the intermediary fuses these scores through weighted combinations to produce a final determination. This intermediary layer simplifies the integration of multiple sensors by providing a standardized interface for combining their outputs, thereby reducing the complexity burden of having multiple sensor types
3Reliability
If confidence-based processing is implemented, then reliability of gesture recognition is improved, but processing time is worsened
Solution Approach 1:
The system implements partial processing by calculating confidence scores selectively based on detection needs. When a gesture is clearly detected by one sensor type with high confidence, the system can forgo extensive processing from other sensor types, using only the necessary subset of sensor data. This partial action approach maintains high reliability by processing enough data to ensure accurate recognition while avoiding unnecessary processing time expenditure on redundant sensor readings when confidence is already sufficient
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables a contextual and delightful user experience by providing relevant information and controls without user intervention, adapting to proximity and gesture detection with high accuracy, enhancing user interaction without disrupting regular activities.
Implementation Method 1
identifying a first sensor signal that can be used to determine a proximity of a presence with respect to the electronic device... determining from the first sensor signal the proximity of the presence
Implementation Method 2
proximity of the presence with respect to the electronic device... identifying a second sensor signal that can be used to capture a gesture of the presence
Implementation Method 3
determining from the second sensor signal the gesture associated with the proximity of the presence... capture a gesture of the presence
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
This application is directed to a method for controlling user experience (UX) operations on an electronic device that executes an application. A touchless UX operation associated with the application has an initiation condition including at least detection of a presence and a gesture in a required proximity range with a required confidence level. The electronic device then determines from a first sensor signal the proximity of the presence with respect to the electronic device. In accordance with a determination that the determined proximity is in the required proximity range, the electronic device determines from a second sensor signal a gesture associated with the proximity of the presence and an associated confidence level of the determination of the gesture. In accordance with a determination that the determined gesture and associated confidence level satisfy the initiation condition, the electronic device initializes the touchless UX operation associated with the application.