Passthrough Feed Style Matching for Smoother VR Transitions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The sudden transition between passthrough feeds and VR content in virtual reality environments can cause user discomfort, including sensory contrast, eye strain, and cognitive dissonance due to differences in rendering styles.
Innovation Solution
Customizing passthrough feeds by incorporating VR content features, using custom style filters or style transfer neural networks to match the passthrough feed with the VR content, and overlaying relevant objects of interest, based on user activity and environmental data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If passthrough feed is enabled to capture real-world images, then user awareness of real environment is improved, but visual comfort deteriorates due to rendering style contrast with VR content
Solution Approach 1:
The system applies colorization to the passthrough feed, transforming black and white real-world images into colored images that match the color characteristics of the VR content. This reduces the visual contrast between the two feeds and eliminates eye discomfort caused by the stark difference between colored VR content and grayscale passthrough.
Solution Approach 2:
The system dynamically adjusts rendering parameters of the passthrough feed including color intensity, saturation, and luminance to match the current VR content style. By changing these parameters in real-time based on the VR content being displayed, the system ensures visual harmony between passthrough and virtual elements.
2Speed
If passthrough feed is activated by events, then responsiveness to real-world conditions is improved, but user experience deteriorates due to sudden unexpected transitions
Solution Approach 1:
The system prepares transition effects and rendering style adjustments in advance before activating the passthrough feed. When an event triggers passthrough activation, the system has pre-computed the appropriate visual transformation, allowing for a smooth transition rather than a sudden jarring change. This preliminary preparation maintains responsiveness while ensuring user comfort.
3Stability of the object's composition
If static filters are applied to passthrough feed, then visual consistency is improved, but adaptability to different VR content styles deteriorates
Solution Approach 1:
The system replaces static filters with dynamic, real-time rendering style transfer that adapts to each frame of VR content. Instead of applying a fixed filter once, the system continuously analyzes the current VR content's visual characteristics and adjusts the passthrough feed's rendering parameters accordingly, maintaining both consistency within each scene and adaptability across different VR content styles.
Data Source
AI summary
System and methods for customizing passthrough feed are disclosed. Expected VR content and predicted user movements in a real-world environment are determined based on recently rendered VR content, user activity data, and real-world scenes. When passthrough feed is enabled, it may be customized based on the expected VR content and predicted user movements. Customization techniques comprising custom style filters, style transfer neural networks, and adding VR content elements to the passthrough feed display window may be used to customize the passthrough feed to match the style of the VR content. The use of the customization techniques may be based on one of or a combination of a duration during which the passthrough feed is activated, the color intensity difference between the realities, an event or purpose which triggered activation of the passthrough feed, or a size of a display window of the passthrough feed relative to the VR content.