Mixed Reality Capture Metadata for Cross-Device Brightness Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for capturing and replaying stereoscopic videos and mixed reality experiences do not provide accurate and desirable viewing experiences across different lighting conditions and devices.
Innovation Solution
The implementation of metadata with recorded or streaming video content to adjust brightness and color based on original viewing conditions, using statistical information and specific camera configurations, and processors to facilitate consistent playback across immersive and non-immersive devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video content is recorded in original format for immersive devices, then viewing experience on immersive devices is preserved, but viewing experience on non-immersive devices deteriorates due to lighting condition differences
Solution Approach 1:
The video content is segmented into multiple versions: original view for immersive devices and adapted view for non-immersive devices. Each version is processed separately with appropriate brightness adjustments based on metadata indicating original viewing conditions, allowing optimal display on each device type without compromising the other
Solution Approach 2:
Brightness parameters of the video content are changed based on the target viewing device type. The system adjusts brightness levels in the adapted view to compensate for ambient lighting conditions that differ from the original immersive viewing environment, while preserving original brightness characteristics for immersive device playback
2Adaptability or versatility
If brightness is adjusted for non-immersive devices, then viewing experience on non-immersive devices improves, but original viewing experience on immersive devices may be compromised
Solution Approach 1:
The system creates separate adapted and original video streams, applying brightness adjustments only to the adapted stream intended for non-immersive devices. The original stream remains unmodified, ensuring that immersive device users experience the content exactly as captured, while non-immersive device users receive optimized brightness levels
Solution Approach 2:
Metadata containing information about original viewing conditions acts as an intermediary that guides the brightness adjustment process. This metadata enables the system to make informed adjustments for non-immersive devices without losing the original viewing characteristics needed for immersive device playback
3Adaptability or versatility
If multiple viewing formats are maintained, then device compatibility improves, but system complexity increases
Solution Approach 1:
Brightness adaptation is performed in advance during content preparation rather than in real-time during playback. The adapted view is pre-processed with appropriate brightness adjustments based on metadata, reducing computational complexity during actual viewing while still providing multiple optimized formats for different devices
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that incorporate enhanced metadata into spatial and/or mixed reality capture environments. For example, a process may include capturing an original video content recording that includes one or more frames depicting a first view provided by a head mounted device (HMD) at one or more instants in time. The process may further generate an adapted video content recording by adapting brightness of the one or more frames of the original video content recording to account for a difference between a first viewing condition associated with the first view and a second viewing condition associated with non-immersive viewing. The process may further generate metadata comprising information associated with the first viewing condition and the metadata may be associated with the adapted video content recording.