Remote GUI Video Stream Separation for Server Load Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In remote desktop technologies, the existing methods for transmitting graphical user interface (GUI) updates between a server and a client often result in high server computing load and network resource utilization, particularly due to the need to decode and re-encode video streams within the server, which can be inefficient and may not account for client performance capabilities.
Innovation Solution
The system renders display updates for a GUI, excluding video streams, and encodes these updates for transmission to the client, allowing the client to decode and display the video stream separately, while optionally modifying or discarding the video stream based on client performance capabilities to reduce network and computing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the server decodes and re-encodes video streams within the GUI, then the video can be displayed within the desktop interface, but the server computing load and network resource utilization increase significantly
Solution Approach 1:
The patent extracts the video stream from the main GUI encoding process. Instead of decoding and re-encoding video within the server's GUI pipeline, the system separates video transmission into an independent channel. The server transmits the encoded video stream directly to the client without processing it through the VGPU stack, thereby reducing server computing load while maintaining video display capability.
Solution Approach 2:
The patent segments the GUI transmission and video stream transmission into separate channels. The GUI updates are encoded and transmitted through the standard remote desktop protocol, while the video stream is transmitted separately as encoded data. This segmentation allows the client to handle video decoding independently, reducing the server's processing burden.
2Ease of operation
If the server transmits the entire GUI including decoded video stream, then the client can display everything uniformly, but the network bandwidth consumption increases
Solution Approach 1:
The patent extracts the video stream from the main GUI data flow and transmits it separately. Instead of including decoded video frames within the GUI bitmap transmission, the system sends encoded video data through a separate channel. This extraction reduces the total amount of data transmitted over the network while maintaining display uniformity through client-side composition.
Solution Approach 2:
Instead of the server decoding video and including it in the GUI transmission (traditional approach), the patent inverts the approach by transmitting encoded video separately and letting the client perform the decoding. This inversion reduces network bandwidth consumption by avoiding the transmission of large decoded video frames within the GUI data stream.
3Stability of the object's composition
If the server processes all display updates including video, then consistent encoding can be maintained, but the processing time and resource usage increase
Solution Approach 1:
The patent extracts video processing from the server's GUI encoding pipeline. By separating video stream handling from the main display update process, the server avoids the time-consuming operations of decoding and re-encoding video frames. The encoding consistency is maintained through the use of standardized encoding formats, while processing time is reduced by eliminating redundant video processing steps at the server.
Data Source
AI summary
Embodiments enable display updates other than a video stream in a graphical user interface (GUI) to be rendered, encoded, and transmitted exclusive of the video stream. A virtual machine generates a GUI that includes an encoded video stream and other display updates. A virtual graphics processing unit (VGPU) stack associated with the VM renders the other display updates of the GUI to a framebuffer. The rendered display updates are encoded and transmitted to a client for display. The encoded video stream, or a modified (e.g., reduced bit rate) version of the encoded video stream, may be transmitted to the client, such that the client can display the encoded video stream within the GUI. For example, the encoded video stream may be selectively transmitted to the client based on the performance capabilities of the client.