Client-Side Remoting Frame Buffering for Bandwidth-Constrained Visual Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Display remoting technologies face challenges in maintaining user experience when operating in environments with limited network bandwidth and CPU resources, leading to degraded performance with fuzzy images and jerky animations due to reduced frame rates and aggressive compression.
Innovation Solution
A client-side system that adaptively applies image enhancement techniques such as frame prediction, interpolation, and artifact reduction based on available network bandwidth and frame quality, without consuming additional network bandwidth, to improve user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If aggressive compression techniques are applied to reduce network bandwidth consumption, then network bandwidth usage is reduced, but image quality deteriorates resulting in fuzzy images and visual artifacts
Solution Approach 1:
The system performs preliminary actions by buffering multiple frames before rendering. This allows the rendering system to work with pre-fetched frame data, enabling more flexible rendering decisions that can improve image quality without requiring additional network bandwidth. The buffered frames provide a pool of data that can be selectively processed to enhance visual output.
Solution Approach 2:
The patent introduces a temporal dimension by utilizing multiple buffered frames across different time points. Instead of processing single frames in isolation, the system leverages temporal information from multiple frames to improve rendering quality. This dimensional approach allows for techniques like motion compensation and temporal filtering that enhance image quality without increasing instantaneous bandwidth requirements.
2Use of energy by moving object
If frame rate is reduced to conserve CPU resources, then CPU resource consumption is reduced, but user experience deteriorates resulting in jerky animations
Solution Approach 1:
The system pre-fetches and buffers multiple frames in advance, allowing the rendering pipeline to work with prepared data. This preliminary action enables the system to maintain smoother visual output by having frames ready for display, reducing the computational burden during actual rendering and improving perceived frame rate without proportionally increasing CPU usage.
Solution Approach 2:
The patent creates copies of frame data in buffer memory, allowing the rendering system to work with these copies rather than repeatedly accessing or re-processing original network data. This copying mechanism enables more efficient rendering operations and allows for quality-enhancing processing without linearly increasing CPU resource consumption.
3Adaptability or versatility
If aggressive compression is applied to meet resource limitations, then the system conforms to available resource constraints, but visual artifacts increase degrading user experience
Solution Approach 1:
By buffering frames in advance, the system gains flexibility in how compression is applied. The preliminary buffering allows for more sophisticated compression strategies that can reduce artifacts, such as selecting optimal compression levels for different frame types or regions, while still meeting overall resource constraints.
Solution Approach 2:
The system applies different rendering qualities to different regions or types of frames based on their importance and content characteristics. This local quality approach allows critical visual areas to maintain higher quality with fewer artifacts, while less important areas can use more aggressive compression, overall conforming to resource constraints while minimizing perceptible visual artifacts.
Data Source
AI summary
The disclosure herein describes a client-side system that enhances user experience on a remoting client without consuming additional network bandwidth. During operation, the system receives a sequence of frame updates for a display screen, and determines a sequence of frames corresponding to the frame updates. The system further adaptively applies one or more image enhancing techniques to the sequence of frames based on available network bandwidth, frame refresh rate, or image quality. The image enhancement techniques include predicting a frame based on previously received frames, interpolating a frame based on at least two buffered frames, and reducing appearance of artifacts in a received frame, thereby reducing visual artifacts.