Post-Capture Artificial Reality Effect Editing via State Information
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Legacy artificial-reality recording solutions cannot change or modify artificial-reality effects after a video stream is recorded, and adding effects to post-capture video streams is limited due to the lack of artificial-reality state information and contextual data, making dynamic editing and rendering of effects impossible.
Innovation Solution
Capturing artificial-reality state information and contextual data streams separately from the video stream, allowing for richer post-capture editing and rendering of artificial-reality effects without regenerating or re-capturing data, enabling identical or new effects to be applied to the video stream.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If artificial-reality effects are fused into the recorded video stream, then the video stream contains the effects, but the effects cannot be changed after recording
Solution Approach 1:
The patent separates the video stream into two independent components: the base video stream and the artificial-reality state information stream. This segmentation allows the video stream to remain stable while the state information stream enables flexible post-capture editing of effects. The state information stream contains parameters that describe the effects applied to each frame, which can be modified independently without affecting the original video data.
2Adaptability or versatility
If post-capture image processing is used to add effects to video streams, then effects can be added, but certain effects are not feasible due to lack of state information
Solution Approach 1:
The patent captures and stores the artificial-reality state information stream during the original video recording process, before any post-capture editing occurs. This preliminary capture of state information (including effect parameters, randomness data, and contextual data) ensures that all necessary information is available for future effect rendering and editing, eliminating the limitations of post-capture image processing.
3Adaptability or versatility
If artificial-reality state information is captured separately from the video stream, then dynamic editing is enabled, but data storage requirements increase
Solution Approach 1:
The patent captures the artificial-reality state information as a separate data stream that represents the effects applied to the video stream. Rather than storing multiple versions of the video with different effects, the system stores a single compact state information stream that can be used to generate or modify effects. This copying approach significantly reduces storage requirements compared to storing multiple full video streams.
4Device complexity
If effects are rendered on post-captured video streams without state information, then processing is simpler, but rendering consistency across devices is compromised
Solution Approach 1:
The patent captures and stores specific parameters in the artificial-reality state information stream that define the effects applied to each frame of the video stream. These parameters include effect type, intensity, timing, and randomness data. By preserving these parameters, the system ensures that effects can be consistently reproduced across different devices and platforms, maintaining reliability while allowing for flexible post-capture editing.
Data Source
AI summary
In one embodiment, a method includes retrieving a video stream that was recorded while a first artificial-reality effect was being displayed on the video stream, where each frame of the video stream comprises a real-world scene without the first artificial-reality effect, retrieving an artificial-reality state information stream corresponding to the video stream, where the artificial-reality state information stream comprises state information associated with the first artificial-reality effect, retrieving one or more contextual data streams corresponding to the video stream, where the first artificial-reality effect displayed on the video stream was rendered based on at least a portion of the one or more contextual data streams, rendering a second artificial-reality effect based on at least a portion of the artificial-reality state information stream and a portion of the one or more contextual data streams, and displaying the second artificial-reality effect on the video stream.