Hybrid GPU CPU Encoding for Virtual Desktop Frame Remoting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current remote display protocols for virtual desktop infrastructure either rely solely on software encoding by the CPU, which is inefficient for 3D and video applications, or solely on hardware encoding by the GPU, which is inflexible for static UI elements, leading to suboptimal performance in delivering virtual desktop frames to client devices.
Innovation Solution
A hybrid approach that combines hardware encoding by the GPU with software encoding by the CPU, where portions of the frame data are encoded in hardware and others in software, and transmitted as separate video streams to optimize performance based on the type of content displayed on the virtual desktop.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If software encoding by CPU is used, then flexibility for static UI elements is maintained, but encoding speed and efficiency for 3D and video applications deteriorates
Solution Approach 1:
The patent divides the frame data into different portions based on content type (3D/video portions vs. static UI portions) and processes each portion through different encoding paths - GPU encoding for 3D/video and CPU encoding for static UI elements. This segmentation allows each component to operate in its optimal performance zone, resolving the contradiction between speed and flexibility.
Solution Approach 2:
Different encoding methods are applied to different local regions of the frame data based on their specific characteristics. The patent identifies which portions are 3D/video content requiring hardware acceleration and which are static UI elements benefiting from software flexibility, then applies the appropriate encoding quality and method to each local region independently.
2Productivity
If hardware encoding by GPU is used, then encoding speed for 3D and video applications is improved, but flexibility for static UI elements deteriorates
Solution Approach 1:
The patent segments the encoding task by content type, routing 3D and video portions to GPU hardware encoding for speed optimization, while routing static UI element portions to CPU software encoding for flexibility. This segmentation resolves the contradiction by allowing each method to shine in its appropriate domain.
Solution Approach 2:
The patent applies different encoding qualities and methods to different local portions of the frame based on their content characteristics. Hardware encoding is applied locally to 3D/video portions where speed is critical, while software encoding is applied locally to static UI portions where adaptability is needed.
3Adaptability or versatility
If CPU encoding is used for all frames, then flexibility and adaptability are maintained, but overall encoding efficiency and performance deteriorates
Solution Approach 1:
The patent merges the GPU hardware encoding path and CPU software encoding path into a unified hybrid encoding system. Both paths process different portions of frame data simultaneously and their outputs are combined into a single complete encoded frame. This merging allows the system to achieve both high efficiency (from GPU) and high flexibility (from CPU) that neither could achieve alone.
Solution Approach 2:
The hybrid encoding system is universal in that it can handle multiple content types within a single frame - 3D video, static UI elements, and transitions between them. The system automatically adapts which encoding method to use for each portion, making it universally applicable to diverse virtual desktop scenarios without sacrificing either efficiency or flexibility.
4Power
If GPU encoding is used for all frames, then encoding efficiency is improved, but adaptability and flexibility for different content types deteriorates
Solution Approach 1:
The patent implements a dynamic encoding system that automatically adjusts which encoding method (GPU or CPU) is applied to each frame portion based on real-time content analysis. The system dynamically routes 3D/video content to GPU encoding and static UI content to CPU encoding, creating an adaptive system that maintains high efficiency while being versatile enough to handle different content types appropriately.
Data Source
AI summary
Frames of a virtual desktop are encoded using a hybrid approach that combines the strength of software encoding by a central processing unit (CPU) and hardware encoding by a graphics processing unit (GPU). A method of encoding frame data of one or more virtual desktops in hardware and in software and transmitting the encoded frame data to one or more client devices, includes the steps of encoding a first portion of the frame data in the GPU to generate a first encoded frame data, encoding a second portion of the frame data in software, i.e., programmed CPU, during encoding of the first portion, to generate a second encoded frame data, and transmitting the first encoded frame data and the second encoded frame data from a host computer of the one or more virtual desktops to the one or more client devices as separate video streams.