Multi-Sensory Interaction Feedback for Immersive AR and VR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality and virtual reality systems lack responses such as body temperature, heartbeat, pulse, smell, taste, and human brain waves, resulting in a poor interactive experience, and the global job shortage exacerbates this issue.
Innovation Solution
A multi-sensory human-computer interaction system and method that utilizes a variety of sensors and output devices, including visual, auditory, tactile, olfactory, gustatory, and brain-computer interface sensors, to generate multi-sensory feedback and assist users in real-world situations through augmented, virtual, and mixed reality technologies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional augmented reality and virtual reality systems are used, then the system structure is simple, but the interactive experience is poor due to lack of multi-sensory feedback
Solution Approach 1:
The patent combines multiple types of sensors (visual, auditory, tactile, olfactory, gustatory, brain-computer interface sensors) and output devices into a unified multi-sensory human-computer interaction system. This merging of diverse sensing and actuation components enables comprehensive multi-sensory feedback, transforming the interactive experience from limited visual-auditory-tactile feedback to a holistic multi-sensory engagement that includes smell, taste, and direct neural interaction.
Solution Approach 2:
The system is designed with universal multi-functionality by incorporating sensors and output devices that cover all six human senses plus brain-computer interface capabilities. Each sensor type and output device can independently detect or stimulate a specific sensory modality, allowing the system to adaptively provide appropriate multi-sensory feedback for diverse interaction scenarios, thereby enhancing interactive experience quality across different application contexts.
2Reliability
If multiple sensory sensors and output devices are integrated, then the interactive experience is enhanced, but the device complexity increases
Solution Approach 1:
The patent segments the multi-sensory system into distinct functional modules: visual sensors, auditory sensors, tactile sensors, olfactory sensors, gustatory sensors, brain-computer interface sensors, and corresponding output devices for each sensory modality. This segmentation allows each sensor type and output device to be independently designed, selected, and optimized for its specific sensory function, making the overall complex system manageable through modular architecture while maintaining high interactive experience quality.
3Measurement precision
If conventional sensors are used, then the device complexity is low, but the measurement precision of physiological parameters is insufficient
Solution Approach 1:
The patent employs parameter changes by selecting and integrating sensors with optimized detection parameters for each specific physiological parameter. For example, temperature sensors are chosen with appropriate sensitivity ranges for body temperature detection, and brain-computer interface sensors are selected with specific neural signal detection capabilities. This parameter optimization ensures high measurement precision for each physiological parameter while managing device complexity through targeted sensor selection rather than comprehensive sensor arrays.
Data Source
AI summary
A multi-sensory human-computer interaction system includes an interactive device, sensors and a server. The server is coupled to the sensors and is configured to control the interactive device. The multi-sensory human-computer interaction system performs the following operations: each of the plurality of sensors receives a plurality of sensing signals from a real environment; the server generates a plurality of event data corresponding to each of the plurality of sensing signals according to the plurality of sensing signals, wherein a plurality of event data represents the situation of the real environment; the server generates a plurality of multi-sensory correlation data corresponding to the situation according to the plurality of event data; the server determines a sensory feedback result according to the plurality of multi-sensory correlation data; and the interactive device is configured to generate interactive messages according to sensory feedback result. The interactive messages are configured to interact with users.