Selective Peripheral Display Rendering via Saliency Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional mixed-reality computer systems are limited by their use of sparse peripheral displays, which impose unnecessary burdens on the system's operations and can cause discomfort due to unnatural or distracting low-resolution displays, and fail to effectively utilize peripheral displays in a 'smart' manner.
Innovation Solution
The system selectively renders peripheral visualizations based on attributes of a computer-generated scene, determining salient elements and their saliency magnitude to activate only necessary pixels, reducing power consumption and improving the natural feel of the mixed-reality environment by dynamically controlling the peripheral displays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If sparse peripheral displays are activated continuously to provide contextual awareness, then the user's immersion and contextual understanding are improved, but power consumption increases and user comfort deteriorates due to unnatural low-resolution displays
Solution Approach 1:
The peripheral display dynamically adjusts its activation state between active and inactive based on real-time saliency detection of scene elements. The display transitions from a static continuous activation mode to a dynamic selective activation mode, activating only when salient elements are detected in the peripheral region.
Solution Approach 2:
Instead of continuously activating the entire peripheral display, the system applies partial action by activating only the specific peripheral region where a salient element is detected. This reduces overall power consumption while maintaining contextual awareness where needed.
2Loss of information
If sparse peripheral displays are activated continuously to provide contextual awareness, then the user's immersion and contextual understanding are improved, but user comfort deteriorates due to unnatural low-resolution displays
Solution Approach 1:
The system dynamically controls the peripheral display activation based on scene saliency, transitioning from continuous static activation to conditional dynamic activation. This reduces exposure to unnatural low-resolution content, improving user comfort while maintaining contextual awareness when necessary.
Solution Approach 2:
The peripheral display is activated partially and selectively only in regions containing salient elements, rather than continuously across the entire periphery. This minimizes the harmful visual impact of low-resolution displays on user comfort.
3Loss of information
If all pixels in the peripheral display are activated to ensure complete scene coverage, then comprehensive scene context is provided, but system complexity and processing burden increase
Solution Approach 1:
The system extracts and identifies only the salient elements from the complete scene using saliency detection algorithms. Instead of processing and displaying all scene elements in the peripheral region, only the most important elements are selected for peripheral display activation, reducing processing burden.
Solution Approach 2:
The peripheral display applies local quality by providing high attention resources (activation) only to specific local regions containing salient elements, rather than uniformly activating the entire peripheral display. This optimizes system resources while maintaining essential scene context.
Data Source
AI summary
Peripheral visualizations are based on various attributes associated with a scene. Characteristics of elements in a scene are determined. Based on these characteristics, the salience of the elements is determined. When the element is salient, then this determination also includes a saliency magnitude of the element. Thereafter, the embodiments determine whether the element's saliency magnitude exceeds a particular saliency threshold. If the magnitude does exceed this threshold, then the embodiments render a corresponding peripheral visualization with the peripheral display(s) proximate the salient element(s).