HMD Situational Awareness via Outward Sensor Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users of head-mounted devices (HMDs) during Computer Generated Reality (CGR) experiences have reduced awareness of their physical environments, leading to unintended impacts on others, such as noise levels or distracting behaviors, due to the immersive nature of CGR which can obscure or replace real-world sensory inputs.
Innovation Solution
The HMD employs outward-facing sensors, such as cameras and microphones, to detect emotional states and reactions of people in the physical environment, providing visual or audio warnings to the user when their actions are likely to annoy or distract others, and adjusts the CGR experience accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If CGR content is provided to replace or obscure real-world sensory inputs, then immersion and engagement in the virtual experience are improved, but awareness of the physical environment and social surroundings deteriorates
Solution Approach 1:
The system continuously monitors the physical environment through outward-facing sensors and provides real-time feedback to the user about their social impact. When the system detects that the user's actions (loud speaking, gesturing, audio playback) are negatively affecting others, it delivers feedback notifications to alert the user and prompt behavior modification, thereby restoring social awareness without completely breaking the CGR immersion.
Solution Approach 2:
The patent introduces an intermediary system (the HMD's sensor suite and processing unit) that mediates between the user's CGR experience and the physical social environment. This intermediary monitors physical surroundings through cameras and microphones, analyzes social context, and translates physical environmental information into meaningful notifications for the user, bridging the gap between virtual immersion and physical awareness.
2Loss of information
If the HMD monitors and analyzes the physical environment to detect emotions and reactions of others, then situational awareness and social sensitivity are improved, but device complexity and processing requirements increase
Solution Approach 1:
The system segments the monitoring task into distinct functional modules: outward-facing cameras capture visual data, microphones capture audio data, a processor analyzes facial expressions and emotions, and another component determines attention direction. This segmentation allows each module to specialize in specific detection tasks, improving overall efficiency while managing complexity through modular architecture.
Solution Approach 2:
The HMD integrates multiple functions into a single device: it provides CGR content delivery, captures physical environment data through outward-facing sensors, analyzes emotions and attention, and delivers feedback notifications. This multi-functionality consolidates what would otherwise require separate systems into one unified device, managing complexity through integration rather than proliferation of separate components.
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods for providing situational awareness for HMD users. Some implementations involve providing a CGR experience to a first user wearing an HMD in a physical environment. Audio data or image data is obtained from a sensor at the physical environment during the CGR experience and an emotional state or direction of attention of a second person other than the first user is detected based on the audio data or image data. A notification or other change is provided in the CGR experience based on detecting the emotional state or direction of attention of the second person.