Mixed Reality Occlusion Filtering for Visible-Region Video Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality systems fail to optimize video quality by accounting for virtual object occlusions, leading to suboptimal brightness, focus, and color balance in regions obscured by virtual content.
Innovation Solution
Implement systems that filter data from Automatic Exposure Control, Automatic White Balance, and Automatic Focus algorithms to exclude regions occluded by virtual objects, adjusting image optimization based on visible areas only.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire video frame is used for optimization calculations, then more data is available for processing, but the video quality in visible regions deteriorates due to occluded regions skewing the optimization statistics
Solution Approach 1:
The patent extracts and removes occluded regions from the video frame before performing optimization calculations. By identifying regions obscured by virtual objects and excluding them from the optimization data, the system ensures that brightness, focus, and color balance adjustments are based only on visible regions, thereby improving overall video quality in those regions.
Solution Approach 2:
The patent segments the video frame into occluded and non-occluded regions. This segmentation allows the system to process different regions differently - excluding occluded regions from optimization calculations while maintaining them in the final output, thus preventing skewing of optimization statistics by invisible areas.
2Adaptability or versatility
If virtual objects are added to enhance mixed reality experience, then interactivity and immersion improve, but video quality in occluded regions deteriorates due to incorrect optimization based on hidden areas
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors which regions of the video frame are occluded by virtual objects and adjusts the optimization process accordingly. By feeding back the occlusion status to the optimization algorithms, the system dynamically adapts to maintain accurate image optimization in visible regions while allowing virtual objects to enhance the mixed reality experience.
3Productivity
If automatic exposure control, white balance, and focus algorithms process the entire frame, then processing efficiency is maintained, but visual accuracy in visible regions deteriorates due to influence from occluded areas
Solution Approach 1:
The patent performs preliminary identification and masking of occluded regions before the automatic exposure control, white balance, and focus algorithms process the video frame. By pre-processing the frame to remove or mask occluded areas, the subsequent optimization algorithms work only with visible region data, ensuring visual accuracy without significantly impacting processing efficiency.
Data Source
AI summary
A computer-implemented method for optimizing for virtual content occlusion in mixed reality may include (i) receiving a frame of a passthrough video captured by a sensor of a mixed-reality display, (ii) detecting at least one region of the frame of the passthrough video that is occluded by a virtual object projected within the mixed-reality display, (iii) creating a modified version of the frame of the passthrough video without the region that is occluded by the virtual object, (iv) providing the modified version of the frame to at least one visual analysis algorithm that calculates at least one statistic based on the modified version of the frame, (v) adjusting the frame by at least one visual adjustment algorithm based at least in part on the at least one statistic, and (vi) displaying the adjusted frame on the mixed-reality display. Various other methods, systems, and computer-readable media are also disclosed.