MPEG File Auxiliary Sensor Data Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video and image encoding formats, such as MPEG, do not efficiently incorporate and utilize additional data streams like accelerometer, compass, lens zoom, aperture, and depth of field information, limiting the ability to process video and audio in a unified and adaptable manner across different devices and environments.
Innovation Solution
Incorporating auxiliary data streams, such as 3D acceleration, orientation, and camera parameters, into a unitary data object like an MPEG file, allowing for synchronized processing and rendering of video and audio, enabling features like motion stabilization and dimensional audio based on user preferences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing video and image encoding formats (e.g., MPEG) are used, then compatibility and simplicity are maintained, but the ability to incorporate and utilize additional data streams (accelerometer, compass, lens parameters) is limited
Solution Approach 1:
The patent combines multiple data streams (video, audio, accelerometer, compass, lens parameters) into a single unified data structure. This merging approach allows all auxiliary data to be incorporated together with the main media content in one encoding framework, resolving the contradiction by enabling versatility while managing complexity through integration rather than separate handling of each data type
Solution Approach 2:
The unified data structure is designed to accommodate multiple types of data (video, audio, sensor data, camera parameters) within a single format. This universal structure can handle diverse data streams simultaneously, improving adaptability while maintaining a consistent encoding approach that doesn't significantly increase complexity
2Adaptability or versatility
If auxiliary data streams are incorporated into the encoding format, then processing versatility is improved, but data structure complexity increases
Solution Approach 1:
The patent segments the unified data structure into distinct components or fields, each dedicated to specific types of data (video segments, audio segments, sensor data segments). This segmentation allows the system to handle complex multi-stream data while organizing it in a structured, manageable way that reduces overall complexity through modular organization
Solution Approach 2:
The patent introduces a new dimensional organization to the data structure by adding temporal and spatial dimensions to how auxiliary data is structured and accessed. This dimensional approach allows versatile processing of multi-stream data while maintaining organized, manageable complexity through structured multi-dimensional indexing and access patterns
3Ease of operation
If audio data is processed in accordance with focal plane data to create dimensional audio effects, then user experience is enhanced, but processing requirements increase
Solution Approach 1:
The patent performs preliminary processing of audio data in conjunction with focal plane information during the encoding phase. By pre-processing and organizing the audio-focal plane relationship data during encoding, the system reduces the computational burden during playback, as the foundational processing work has already been completed and stored in the unified data structure
Data Source
AI summary
Audio and or video data is structurally and persistently associated with auxiliary sensor data (e.g., relating to acceleration, orientation or tilt) through use of a unitary data object, such as a modified MPEG file or data stream. In this form, different rendering devices can employ co-conveyed sensor data to alter the audio or video content. Such use of the sensor data may be personalized to different users, e.g., through preference data. For example, accelerometer data can be associated with video data, allowing some users to view a shake-stabilized version of a video, and other users to view the video with such motion artifacts undisturbed. In like fashion, camera parameters, such as focal plane distance, can be co-conveyed with audio/video content—allowing the volume to be diminished (or not, again depending on user preference) when a camera captures audio/video from a distant subject. Some arrangements employ multiple image sensors and/or multiple audio sensors—each also collecting auxiliary data. A great number of other features and arrangements are also detailed.