Mixed Reality Recording Track Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices, such as head-mounted devices, face challenges in recording and replaying mixed reality scenes without artifacts when displayed on non-stereoscopic devices, and editing these recordings is difficult due to the integrated nature of the content.
Innovation Solution
A method for operating electronic devices that captures video frames and renders virtual content, storing a mixed reality scene as separate tracks with metadata, allowing for editing and replay on different devices with adjusted parameters, ensuring compatibility and flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the mixed reality scene is recorded as integrated content, then the recording process is simple, but the recorded content has artifacts when displayed on non-stereoscopic devices and editing is difficult
Solution Approach 1:
The patent segments the mixed reality content into separate tracks: a first track containing video frames captured by cameras, a second track containing rendered virtual content, and metadata tracks containing parameters used during rendering. This segmentation allows each component to be processed and edited independently, eliminating display artifacts on non-stereoscopic devices while enabling flexible editing of individual content layers.
2Device complexity
If the mixed reality scene is recorded as integrated content, then the recording structure is simple, but editing the recording is difficult due to the integrated nature of the content
Solution Approach 1:
The recording is divided into multiple independent tracks (video frames, virtual content, metadata) that can be separately accessed and edited. This allows users to modify specific aspects of the mixed reality scene (such as adjusting virtual content parameters or correcting video frames) without affecting other components, significantly improving editing capability while maintaining an organized structure.
Solution Approach 2:
The patent enables dynamic editing of the recorded mixed reality scene by allowing parameters in the metadata to be modified after recording. Users can adjust rendering parameters, camera settings, and virtual content properties independently, and the system dynamically reapplies these edited parameters during replay to generate modified versions of the original scene.
3Device complexity
If parameters are not separated in the recording, then the file structure is simpler, but the content cannot be adapted to different display conditions
Solution Approach 1:
Parameters used during rendering are separated into distinct metadata tracks, allowing them to be independently accessed and adjusted. This enables the same recorded content to be adaptively displayed on different devices (stereoscopic head-mounted displays, non-stereoscopic screens, VR headsets) by selecting and applying appropriate parameter sets from the metadata for each display type.
Solution Approach 2:
The patent stores multiple parameter variants in the metadata that can be applied during replay based on the target display device. By changing parameters such as stereoscopic rendering settings, field of view angles, and distortion corrections, the system adapts the recorded mixed reality content to match the specific characteristics of different display devices, ensuring optimal display quality across diverse platforms.
Data Source
AI summary
A head-mounted device is provided that includes a variety of subsystems for generating extended reality content, displaying the extended reality content, and recording the extended reality content. The device may include a graphics rendering pipeline configured to render virtual content, tracking sensors configured to obtain user tracking information, a virtual content compositor configured to composite virtual frames based on the virtual content and the user tracking information, cameras configured to capture a video feed, a media merging compositor configured to overlay the composited virtual frames and the video feed, and a recording pipeline configured to record parameters, metadata, raw content, and/or adjusted content in an extended reality recording file. The extended reality recording file may have multiple discrete portions that may each be individually edited. The extended reality recording file may be used to present a replay on the head-mounted device and/or may be exported to an external device.