Client Device Frame Interpolation for Remote Display Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtual desktop infrastructure (VDI) environments, limited hardware resources and network bandwidth can lead to suboptimal user experiences due to reduced frame rates and resolutions, causing fuzzy or jerky animations and video, which degrade the viewing experience for remote desktop users.
Innovation Solution
The use of local computing capabilities on client devices to apply frame interpolation and image upscaling techniques, such as motion estimation and motion compensation (MEMC) and deep learning super sampling (DLSS), to enhance the display quality by generating intermediate frames and increasing resolution, thereby improving user experience without increasing resource consumption at the remote desktop.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the VDI agent reduces display frame rate and resolution to accommodate multiple users on limited hardware resources, then hardware resource consumption is reduced, but user experience degrades due to fuzzy images and jerky animations
Solution Approach 1:
Instead of reducing quality at the remote desktop end to save hardware resources, the patent inverts the approach by applying enhancement techniques at the client device. The VDI agent transmits reduced-resolution frames, and the client device uses local computing power (GPU/CPU) to upscale and interpolate frames, effectively reversing the traditional quality-reduction strategy.
Solution Approach 2:
The patent introduces an intermediary enhancement process between the remote desktop and the display. This intermediary layer uses client device computing capabilities to process the reduced-resolution frames, generating high-quality output without requiring the remote desktop to allocate additional hardware resources.
2Quantity of substance
If the VDI agent reduces display frame rate to accommodate network bandwidth limitations, then network bandwidth consumption is reduced, but user experience degrades due to jerky animations and video
Solution Approach 1:
The patent reverses the traditional approach by not reducing frame rate at the source but rather enhancing frames locally. The VDI agent transmits fewer, lower-resolution frames to conserve bandwidth, and the client device uses local computing power to interpolate and upscale these frames, achieving high frame rates without proportional bandwidth increases.
Solution Approach 2:
The patent changes the parameters of frame processing by applying interpolation and upscaling techniques at the client device. This transforms low-resolution, low-frame-rate input into high-resolution, high-frame-rate output, effectively changing the quality parameters without increasing the transmitted data volume.
3Reliability
If enhancement techniques are applied at the client device to improve display quality, then user experience is improved, but client device computing resources are consumed
Solution Approach 1:
The patent enables the client device to serve itself by using its local computing resources to enhance display quality. The enhancement techniques (interpolation, upscaling) are performed locally on the client device, allowing the user's personal computer to compensate for remote desktop resource limitations without requiring additional infrastructure.
4Productivity
If multiple users share limited hardware resources on a host, then hardware resource utilization is optimized, but display quality and frame rate must be reduced for each user
Solution Approach 1:
The patent inverts the traditional resource allocation strategy by not reducing quality at the remote desktop but rather enhancing it at the client device. This allows multiple users to share hardware resources efficiently while maintaining high display quality through local processing power.
Solution Approach 2:
The patent introduces an intermediary enhancement layer that sits between the shared hardware resources and the display output. This intermediary process uses client device computing capabilities to restore display quality, allowing efficient hardware sharing without quality compromise.
Data Source
AI summary
The disclosure provides a method for optimizing remote display at a client device in communication with a remote desktop. The method generally includes receiving one or more frames for display at the client device, determining that a frame rate of the received one or more frames is less than an expected frame rate and/or a resolution of the received one or more frames is less than an expected resolution, determining that at least one of a local central processing unit (CPU) usage at the client device is less than a CPU usage threshold or a local graphics processing unit (GPU) usage at the client device is less than a GPU usage threshold, applying one or more enhancement techniques to the received one or more frames to produce one or more optimized frames while continuously monitoring the local CPU usage and/or the local GPU usage at the client device, and rendering the one or more optimized frames for display at the client device.