Remote Display Protocol for Video Decoding Offload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In desktop virtualization environments, decoding and transmitting encoded video data consumes significant network bandwidth and computing resources, preventing video players from compositing transparent or semi-transparent UI elements due to the lack of access to decoded video data.
Innovation Solution
Offloading decoding of encoded video data from the virtual machine to a remote display terminal, where the video player decodes and composites graphical overlays, and then transmits the encoded video data and overlays for final decoding and composition on the user's end terminal.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If video data is decoded in the virtual machine, then UI elements can be composited, but network bandwidth and computing resources are consumed
Solution Approach 1:
The patent introduces a remote display protocol as an intermediary that intercepts video decoding requests from the virtual machine and redirects them to the user's local system. This mediator enables the virtual machine to maintain UI compositing capability while avoiding the resource consumption of local decoding, as the actual decoding occurs at the user's end where resources are abundant.
Solution Approach 2:
The patent shifts the video decoding operation from the virtual machine dimension to the user's local system dimension. By moving the decoding process to another dimension (the user's local environment), the system resolves the contradiction by allowing UI compositing in the virtual machine while performing resource-intensive decoding at the user's end, thereby eliminating network bandwidth and computing resource consumption at the server.
2Ease of operation
If video data is decoded at the server, then UI overlays can be rendered, but computing resources are consumed
Solution Approach 1:
The remote display protocol acts as an intermediary that prevents the virtual machine from consuming computing resources for video decoding. Instead, it forwards decoding requests to the user's local system, which has sufficient computing resources to handle the decoding and rendering of UI overlays without impacting the server's computing capacity.
Solution Approach 2:
The patent employs a copy mechanism where the virtual machine receives a copy of the encoded video data and forwards it to the user's local system for decoding. This copying approach allows the virtual machine to maintain the appearance of video playback and UI overlay capability while actually delegating the resource-intensive decoding to the user's system, thereby avoiding server resource consumption.
3Loss of energy
If video decoding is offloaded to the user terminal, then server resources are saved, but UI compositing is prevented
Solution Approach 1:
The remote display protocol serves as an intermediary that preserves UI compositing capability even after video decoding is offloaded. It intercepts the video stream, separates the encoded video data from the UI overlay requests, and forwards both to the user's local system where the UI elements can be composited over the decoded video without requiring server resources.
Solution Approach 2:
The patent segments the video processing tasks into two distinct parts: encoded video data transmission and UI overlay compositing. By segmenting these functions and forwarding them separately to the user's local system, the system eliminates server resource pressure while maintaining full UI compositing capability, as both video decoding and overlay rendering occur at the user's end.
Data Source
AI summary
A method for offloading decoding of encoded video data from a computer system executing a video player application playing the encoded video data to a remote display terminal still enables the video player application to overlay transparent or semi-transparent graphical elements, such elements, such as user interface controls, onto the video, despite not having access to decoded video data. A video decoding function call from the video player application is intercepted and replacement video data is provided to the video player application rather than a decoded version of the encoded video data. The video player application is thereby able to overlay graphical elements onto the replacement video data to create composite video from which the graphical elements are then able to be subsequently extracted and transmitted with the encoded video data to remote display terminal, which decodes the video data and overlays the graphical elements for display.