Speaking Motion Detection for Immersive Headset Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users wearing noise cancellation headphones or head-mounted display devices struggle to recognize surrounding people speaking to them due to auditory or visual immersion, leading to difficulties in communication, especially when the speaker is not in the same direction or has left after speaking.
Innovation Solution
An information processing device that detects and specifies speaking motion information of surrounding persons using imaging and sound monitoring units, allowing for controlled notifications to the user, even when they are auditorily or visually restricted.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If noise cancellation headphones or head-mounted display devices are used to block surrounding information, then the user experiences a sense of immersion, but the user cannot easily understand what surrounding people are saying
Solution Approach 1:
The system performs preliminary detection of speaking motions and pre-processes external sound information before the user needs it. When a speaking motion is detected, the system prepares and stores the corresponding sound information in advance, so that when notification is provided to the user, the information is already ready for immediate presentation, thus resolving the contradiction between maintaining immersion and providing timely external information.
Solution Approach 2:
The system introduces an intermediary mechanism that detects speaking motions of surrounding people and serves as a bridge between the blocked external environment and the user. This intermediary processing unit monitors the surroundings, identifies speaking actions, and selectively transmits relevant sound information to the user without completely breaking the immersion experience, thus mediating between the two conflicting requirements.
2Ease of operation
If external sound information is displayed or output according to the degree of necessity, then communication with outside persons is improved, but the user may still be unable to notice a person who speaks if they are facing a different direction or have already left
Solution Approach 1:
The system segments the detection process into two independent components: speaking motion detection (visual/physical action) and sound information detection (auditory information). By separating these detection functions, the system can identify not only what was said but also the physical context of the speaker's action, providing more complete information for user awareness and response.
Solution Approach 2:
The system merges speaking motion information with external sound information to create a comprehensive notification. By combining visual detection of speaking actions with auditory capture of spoken content, the system provides the user with both the contextual information about the speaker's behavior and the actual sound, ensuring the user can notice and respond to speakers even when directional audio alone would be insufficient.
3Reliability
If the user is notified of external information with a delay after the speaking occurs, then the immersive experience is maintained, but the communication effectiveness is reduced
Solution Approach 1:
The system performs preliminary detection and pre-processing of speaking events before full notification is required. By detecting speaking motions in advance and pre-loading the corresponding sound information, the system minimizes the actual notification delay when the user needs to be informed, thus reducing the loss of time while maintaining immersion during the detection and preparation phase.
Data Source
AI summary
There is provided an information processing device to improve communication between a user and a person speaking to the user by specifying speaking motion information indicating a motion of a surrounding person speaking to the user for whom information from the surroundings is auditorily or visually restricted, the information processing device including: a detecting unit configured to detect a speaking motion of a surrounding person speaking to a user using a device that auditorily or visually restricts information from surroundings; and a specifying unit configured to specify speaking motion information indicating the speaking motion on a basis of monitored surrounding information in a case in which the speaking motion is detected.