Haptic Stream Semantics for Compact Synchronized Media Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The timing model for carrying haptic tracks in multimedia presentations is unclear, and there is a lack of a flexible way to signal semantic descriptions in haptic formats, which affects the compactness and flexibility of haptic signaling.
Innovation Solution
A method for encoding and decoding haptic experiences using a processor to determine the scheme for semantic description of effects in haptics interchange and streaming formats, indicated by specific syntaxes such as 'MPEG_haptics.perception object' and 'readMetadataPerception()', which manage the decoding process based on the format type.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a semantic description is added to haptics interchange format, then the flexibility of signaling semantic descriptions is improved, but the compactness of the format deteriorates
Solution Approach 1:
The patent introduces a syntax parameter 'effectSemantic' that can take different values to indicate the presence and type of semantic descriptions. By changing this parameter, the system can flexibly enable or disable semantic signaling without altering the fundamental structure of the interchange format, thus maintaining compactness while providing adaptability when needed.
Solution Approach 2:
The patent makes the semantic description structure dynamic by using conditional syntax elements. The 'hasSemantics' flag dynamically determines whether semantic keywords are present, allowing the format to adapt its structure based on the specific haptic effect being described, rather than requiring static allocation of space for all possible semantics.
2Adaptability or versatility
If multiple semantic keywords are included in haptics streaming format, then the semantic description capability is improved, but the data transmission overhead increases
Solution Approach 1:
The patent implements partial action by using optional semantic keywords rather than requiring complete semantic descriptions for all haptic effects. The 'hasSemantics' flag and conditional 'effectSemantic' elements allow the system to include only the necessary semantic information for each specific effect, avoiding unnecessary data transmission overhead while maintaining the capability to provide comprehensive semantic descriptions when needed.
3Reliability
If a timing model is established for haptics tracks, then the synchronization with other media tracks is improved, but the complexity of the timing model increases
Solution Approach 1:
The patent applies universality by using the existing ISOBMFF timing infrastructure for haptics tracks rather than creating a separate timing model. The haptics track timing is synchronized with other media tracks through the common ISOBMFF presentation time stamp (PTS) and decomposition time stamp (DTS) mechanisms, allowing the same timing system to serve multiple media types without increasing complexity.
Data Source
AI summary
Method, apparatus, and system for haptic signal processing are provided. The process may include receiving a media stream comprising at least one haptic track and at least one video track. The process may include receiving a media stream, in a case that the data is in the haptics interchange format, obtaining a first syntax, from the media stream, and determining, from the first syntax, a scheme used for a semantic description of effects of the media stream, in a case that the data is in the haptics streaming format, obtaining a second syntax, from the media stream, and determining, from the second syntax, the scheme used for a semantic description of effects of the media stream and a number of characters in the scheme, and controlling decoding of the media stream based determining the scheme.