Wearable Teleconferencing System with Eye Tracking and Physiological Sensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current teleconferencing systems for remote assistance lack the ability to effectively determine the local user's focus and emotional state, leading to inappropriate or timed assistance, which can frustrate both local and remote users due to limited non-verbal cues and increased cognitive load.
Innovation Solution
A teleconferencing system incorporating a wearable audio-visual module with a head-mounted display, eye tracking, and physiological sensors to provide a view from the local user's perspective, track eye gaze direction, and monitor physiological parameters, transmitting this data to the remote user for enhanced understanding and timely assistance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a camera is used to provide a view from the local user's perspective, then the remote expert can see what the user is viewing, but the remote expert cannot determine what the user is actually focusing on or pick up visual clues
Solution Approach 1:
The patent combines multiple sensing modalities (eye tracking, physiological sensors, audio analysis, video feed) into a unified teleconferencing system. The eye tracking system merges with the camera system to provide both the user's perspective and gaze direction, while physiological sensors merge with audio-visual data to comprehensively assess the user's emotional state, thereby resolving the information loss without proportionally increasing complexity
Solution Approach 2:
The patent introduces an intermediary processing system that analyzes eye tracking data, physiological signals, and audio-visual inputs to infer the user's emotional state and attention focus. This intermediary layer translates raw sensor data into meaningful information about user state, enabling the remote expert to understand user focus and emotions without directly observing facial expressions or body language
2Ease of operation
If teleconferencing systems provide remote assistance without emotional state information, then the system is simpler to implement, but assistance may not be appropriately timed which can annoy or frustrate users
Solution Approach 1:
The patent implements feedback loops where physiological sensors continuously monitor the user's emotional state (heart rate, skin conductance, temperature) and provide real-time information to the remote expert. This feedback enables the remote expert to adjust their assistance timing and approach based on the user's current emotional state, improving collaboration quality while the automated sensing reduces the burden on users to manually report their state
Solution Approach 2:
The system performs preliminary assessment of the user's emotional state through physiological sensors before the remote expert provides assistance. By detecting emotional cues in advance through biometric data, the system prepares the remote expert to provide appropriately timed and styled assistance, preventing user frustration before it occurs
3Loss of information
If the camera is directed outward facing at the task, then the remote expert can see the task being performed, but the remote expert cannot see the user's face to pick up non-verbal cues
Solution Approach 1:
The patent makes the teleconferencing system multi-functional by integrating multiple sensing capabilities into a single unified system. The wearable device simultaneously captures the user's perspective via camera, tracks eye gaze direction, monitors physiological parameters, and records audio, replacing multiple separate systems with one universal platform that provides comprehensive situational awareness
Data Source
AI summary
A teleconferencing system for providing remote assistance comprises a local user apparatus that captures both the field of view of the local user along with physiological state of the local user, and a remote user apparatus to allow the remote user to view what the local user is seeing and doing. The local user and remote user have a shared view and a shared audio link, and the remote user can provided assistance to the local user in performing tasks at the local site. Additionally the physiological state of the local user is monitored, and from the physiological state data, an estimate of the emotional state or physiological state of the local user is provided to the remote user. The remote user can interact or assist the local user, including controlling the amount and/or type of assistance provided to the local user based on their current emotional state.