Remote Display Resource Management via Eye Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions for managing computing resources in remote desktop applications are inefficient, leading to resource waste due to idle time limitations that disconnect users unexpectedly, causing frustration and additional resource usage when idle times are set longer to prevent disconnection.
Innovation Solution
A system and method for dynamic remote display resource management that uses eye tracking to monitor user attention on dynamic content, allowing for the downgrading of resource-intensive content when the user is not looking at it, such as reducing frame rates or freezing video, and restoring quality when the user returns, thereby optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If idle time limits are set to prevent resource waste, then resource consumption is reduced, but users may be disconnected unexpectedly causing frustration
Solution Approach 1:
The system dynamically adjusts the idle time threshold based on real-time monitoring of graphical data changes. When the remote desktop is static, a shorter idle time is applied to save resources; when dynamic content is detected, the idle time is extended to prevent disconnection. This dynamic adjustment resolves the contradiction by making the idle time parameter adaptive rather than fixed.
Solution Approach 2:
The system implements feedback by continuously monitoring graphical data for changes and using this information to adjust idle time settings. The monitoring module detects whether the display content is static or dynamic, and this feedback loop allows the system to automatically adjust resource allocation and idle time thresholds, preventing both resource waste and unexpected disconnections.
2Ease of operation
If longer idle times are set to prevent disconnection, then user experience is improved, but additional resource waste occurs while machines sit idle
Solution Approach 1:
The system applies different idle time thresholds dynamically based on display activity. When graphical data shows static content, a shorter idle time (e.g., 5 minutes) is applied to conserve resources. When dynamic content is detected, the idle time is extended (e.g., 30 minutes or longer) to improve user experience. This resolves the contradiction by making idle time adaptive to actual usage patterns.
Solution Approach 2:
The system monitors different regions of the graphical data independently to detect dynamic content. By identifying which specific portions of the display are changing, the system can apply localized quality adjustments and differential idle time settings to different screen regions, optimizing resources while maintaining user experience where needed.
3Ease of operation
If dynamic content is maintained at high quality, then user experience is preserved, but CPU, GPU, memory, and bandwidth resources are consumed
Solution Approach 1:
The system applies partial action by selectively downgrading only the portions of dynamic content that are not currently being viewed. The monitoring module detects user gaze direction and reduces quality (frame rate, resolution, bitrate) for content outside the user's field of view, while maintaining high quality for content in the current view. This partial application of quality reduction saves resources while preserving user experience for visible content.
Solution Approach 2:
The system changes multiple parameters of dynamic content based on viewing status, including frame rate, resolution, bitrate, and encoding quality. When content is not being viewed, these parameters are reduced to lower resource consumption. When content enters the user's view, parameters are restored to maintain quality. This parametric control allows flexible resource management while preserving user experience.
4Loss of energy
If eye tracking is implemented to monitor user attention, then resource optimization is enabled, but device complexity increases
Solution Approach 1:
The system uses an intermediary approach by first detecting cursor position and graphical data changes as proxies for user attention, rather than immediately implementing full eye tracking hardware. This intermediate layer of abstraction allows the system to achieve resource optimization using existing, simpler components. Eye tracking functionality can be added later as an enhancement without requiring fundamental system redesign.
Solution Approach 2:
The system creates a virtual model of user attention by analyzing cursor movement patterns and graphical data changes, which serves as a copy or representation of actual user gaze. This virtual attention model allows the system to optimize resources based on inferred user focus without requiring physical eye tracking devices, thereby reducing device complexity while achieving similar resource optimization goals.
Data Source
AI summary
Systems and methods for remote display resource management can allow a remote display to be dynamically modified according to user eye movements. A user device can receive graphical data for a remote display and present the remote display on a local display. A remote display module (RD module) can detect dynamic content in the graphical data. The RD module can monitor eye movements of a user using an eye tracker device to determine when a user is not looking at the dynamic content. When a user looks away from the dynamic content for more than an allowed amount of time, the RD module can send instructions to a server to reduce the quality of the dynamic content in the remote display. The RD module can send instructions to restore the quality of the dynamic content after determining that the user is looking at the dynamic content.