Dynamic Encoding Adjustment for Screen Streaming Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing screen data streaming technologies often result in degraded user experience due to insufficient client rendering performance, leading to frame drops and wastage of network resources, especially in environments where delays are not tolerated, such as virtual desktops and video conferencing.
Innovation Solution
The system dynamically adjusts the encoding method based on client rendering times by tracking frame rendering performance, modifying encoding parameters or switching encoders to improve client rendering efficiency, such as reducing quality or resolution, to ensure timely and efficient delivery of screen data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the server streams screen data at a high frame rate to improve user experience, then the quality and responsiveness of the displayed content is improved, but the client device may lack sufficient computing resources to decode frames at the required rate, leading to frame drops and degraded performance
Solution Approach 1:
The patent implements dynamic encoding method modification where the server adjusts encoding parameters based on real-time client rendering performance feedback. The system transitions from a static encoding approach to a dynamic one that adapts to changing client capabilities, allowing the frame rate and quality to be optimized based on actual client performance conditions.
Solution Approach 2:
The system incorporates a feedback mechanism where the client reports frame rendering times and performance metrics back to the server. The server uses this feedback to determine when to modify encoding methods, creating a closed-loop control system that continuously optimizes the balance between frame rate delivery and client rendering capability.
2Manufacturing precision
If the server sends screen data at optimal quality to improve user experience, then the visual quality is improved, but network resources are wasted when the client cannot render frames quickly enough to display them
Solution Approach 1:
The server dynamically changes encoding parameters such as quality, resolution, and compression level based on client rendering performance. When the client cannot keep up with the frame rate, the server modifies these parameters to reduce the decoding burden, thereby optimizing the balance between quality and network resource utilization.
Solution Approach 2:
The system initially sends screen data at high quality and frame rate (excessive action), then selectively reduces quality or frame rate (partial action) when client performance metrics indicate that the client cannot render frames fast enough. This prevents network resource waste while maintaining optimal quality when possible.
3Productivity
If the system uses buffering to improve client rendering performance, then the client has more time to decode frames, but applications such as virtual desktops and video conferencing cannot tolerate delays
Solution Approach 1:
Instead of using buffering to improve rendering efficiency, the system changes the encoding parameters to reduce client-side decoding complexity and time requirements. This allows the system to maintain real-time performance while still improving client rendering efficiency through optimized encoding methods adapted to client capabilities.
Data Source
AI summary
Systems and methods are described for providing ways to optimize client performance during screen data streaming. Periods of time a client takes to render frames can be tracked and the frame rendering times can be analyzed to determine when the client performance is insufficient. If the client rendering performance is determined to be insufficient, the encoding method can be dynamically modified in ways preferable for improving the client rendering performance. Different approaches can be utilized for calculating metrics and determining when to modify the encoding method, such as linear interpolation or by taking averages of frame rendering times in a moving sample window.