DASH-ARM Mulsemedia Encoding for Synchronization and Adaptive Delivery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mulsemedia applications face challenges in synchronizing and delivering content that engages multiple human senses, such as sight, sound, touch, and smell, particularly over diverse network environments, and do not support adaptive delivery using widely used protocols like HTTP.
Innovation Solution
A method and system utilizing Dynamic Adaptive Streaming over HTTP (DASH)-based Adaptive Rich Media Delivery (DASH-ARM) for encoding and decoding mulsemedia streams, which adjusts transmission based on network conditions and synchronizes diverse components to create an immersive experience by encoding mulsemedia content into a multi-sensory media presentation description (Mulse-MPD) data stream.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If adaptive delivery is implemented to adjust transmission to network conditions, then user experience quality is improved, but system complexity increases
Solution Approach 1:
The mulsemedia content is segmented into multiple adaptive sets with different complexity levels and sensory component combinations. Each adaptive set contains multiple representations with varying quality and sensory element inclusion, allowing the system to select appropriate segments based on network conditions and device capabilities without requiring complex real-time processing decisions.
Solution Approach 2:
The system pre-processes and organizes mulsemedia content into multiple adaptive sets during encoding, with each set containing pre-configured representations tailored to different network conditions and device capabilities. This preliminary structuring eliminates the need for complex runtime decision-making, reducing system complexity while maintaining adaptive delivery capabilities.
2Reliability
If synchronization of diverse mulsemedia components is implemented, then immersive experience is improved, but processing complexity increases
Solution Approach 1:
The content is divided into periodic structures where each period contains synchronized mulsemedia segments with defined temporal relationships. This segmentation into discrete, pre-synchronized periods simplifies the synchronization process by providing clear temporal boundaries and reference points for aligning multiple sensory components.
Solution Approach 2:
Synchronization metadata and temporal alignment information are pre-calculated and embedded in the Mulse-MPD structure during content creation. This preliminary synchronization preparation eliminates the need for complex runtime synchronization algorithms, reducing processing complexity while maintaining high synchronization accuracy.
3Ease of operation
If HTTP protocol support is implemented for content delivery, then ease of operation is improved, but delivery control flexibility is reduced
Solution Approach 1:
The system uses the universal HTTP protocol as the transport mechanism while embedding DASH-based adaptive streaming logic within the Mulse-MPD structure. This allows the system to leverage HTTP's widespread compatibility and simplicity while maintaining flexible adaptive delivery control through the layered Mulse-MPD specification that works over standard HTTP infrastructure.
Data Source
AI summary
The method and system of the present invention provides Dynamic Adaptive Streaming over HTTP (DASH)-based Adaptive Rich Media Delivery Solution (DASH-ARM) which performs adaptive delivery of mulsemedia content by improving the perceived QoE levels which improves user's sense of reality. In its dynamic adjustment of multiple sensorial content characteristics, DASH-ARM performs a trade-off between video quality and presence of diverse other sensorial components which include components such as haptic, olfaction and wind. This trade-off relies on the fact that the presence of other sensorial components has a masking effect on potential audio/video quality variations. DASH-ARM is complemented by an adaptive multi-sensory media stream (mulsemedia) which synchronizes at the user device, where the multimedia content and mulsemedia comprises diverse sensorial stimuli.