Remote Window GUI Rendering via Local Event Detection

Solution Overview

Problem

Latency in remote desktop sessions leads to poor user experiences during dynamic interactions, such as moving and resizing windows, due to stuttered motion and graphical artifacts.

Innovation Solution

A system and method that generate a graphical user interface for remote applications, allowing for improved window-related operations by detecting mouse input events, determining if they are associated with window move or resize events, and locally caching and updating window positions and images to reduce reliance on remote server updates.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If remote desktop session updates are used for window movement, then window position accuracy is maintained, but user experience degrades due to latency and stuttered motion

Engineering Contradiction:

Improvewindow position accuracyVSAvoiduser experience

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The system performs preliminary actions by detecting mouse input events locally and determining window move events before receiving remote server updates. The client device proactively identifies drag operations and prepares for window movement in advance, rather than passively waiting for remote updates, thereby reducing latency and improving motion smoothness.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system inverts the traditional remote desktop update model by having the client device determine window move events locally rather than relying solely on server-initiated updates. Instead of the server pushing position updates to the client, the client autonomously detects and responds to user interactions, reversing the information flow direction for window positioning.

Inventive Principle:

Principle #13The other way round (Inversion)

2Stability of the object's composition

If remote server updates are awaited for window operations, then graphical consistency is maintained, but motion smoothness deteriorates due to network latency

Engineering Contradiction:

Improvegraphical consistencyVSAvoidmotion smoothness

Core Design Contradiction:

Stability of the object's compositionVSSpeed

Solution Approach 1:

The system performs preliminary actions by detecting mouse input events locally and determining window move events before receiving remote server updates. The client device proactively identifies drag operations and prepares for window movement in advance, rather than passively waiting for remote updates, thereby reducing latency and improving motion smoothness.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system applies dynamics by adapting the window update mechanism based on the operation type. For window movement operations, the client uses local detection and caching to achieve smooth real-time responsiveness. For other operations, the system maintains traditional remote update patterns, dynamically switching between update modes to optimize both smoothness and consistency.

Inventive Principle:

Principle #15Dynamics

3Productivity

If local caching is used for window movement, then responsiveness improves, but reliance on remote updates decreases potentially affecting synchronization

Engineering Contradiction:

ImproveresponsivenessVSAvoidsynchronization

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary actions by detecting mouse input events locally and determining window move events before receiving remote server updates. The client device proactively identifies drag operations and prepares for window movement in advance, rather than passively waiting for remote updates, thereby reducing latency and improving motion smoothness.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by continuously monitoring remote server responses and using them to verify and update locally cached window states. When the remote server sends window move events or updated window images, the client compares these with local cache data and adjusts accordingly, ensuring synchronization while maintaining responsive local operation.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS12242869B1Enhancing user experience on moving and resizing windows of remote applications

Publication Date: 2025.03.04 PARALLELS INT GMBH

AI summary

Disclosed are systems and methods for providing improved rendering of graphical user interfaces for remote applications executing within remote sessions. A remote session client that renders the graphical user interface (GUI) for a remote application is configured to detect certain mouse events that would trigger window move events. In doing so, the remote session client may handle rendering the window GUI locally rather than wait for window move events from the remote application. The remote session client may be further configured to predict using calculated resize patterns how the window content image changes while the remote application's window is being resized.