Network-Aware Split Rendering for Cloud Gaming Frame Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud gaming, varying network bandwidth causes the quality of rendered frames to degrade, leading to visual artifacts and making text in heads-up displays (HUD) difficult to read, which negatively impacts the gameplay experience.
Innovation Solution
A network and render-aware split rendering technique is implemented, where rendering operations are split between cloud gaming servers and clients, with clients executing commands for text, HUD elements, and other game layers, and performing post-processing 3D operations to maintain frame quality across bandwidth variations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If cloud gaming service encodes frames at low bitrate due to varying network bandwidth, then network transmission efficiency is improved, but visual quality and text readability in HUD deteriorate
Solution Approach 1:
The rendering process is segmented into two independent paths: cloud gaming server renders the 3D game scene while client device renders the 2D HUD overlay. This segmentation allows each component to be optimized independently - the server can use lower bitrate for 3D content while the client renders high-quality text and UI elements locally, eliminating the trade-off between bandwidth usage and visual quality.
2Adaptability or versatility
If cloud gaming service reduces encoding quality to adapt to bandwidth limitations, then network transmission is improved, but text readability and HUD clarity deteriorate
Solution Approach 1:
The client device acts as an intermediary that receives the rendered 3D scene from the cloud server and locally generates the 2D HUD overlay. This intermediary approach allows the HUD text and UI elements to be rendered at full quality on the client side without being constrained by network bandwidth, while the server adapts to network conditions by optimizing only the 3D scene transmission.
3Extent of automation
If all rendering operations are performed on cloud gaming server, then centralized control is improved, but network dependency and vulnerability to bandwidth variations increase
Solution Approach 1:
The rendering workload is segmented between server and client: the server handles 3D scene rendering while the client handles 2D HUD rendering. This segmentation reduces network dependency because the client-generated HUD elements do not need to be transmitted over the network, eliminating them from the vulnerable transmission path and improving reliability against bandwidth variations.
Solution Approach 2:
The system dynamically adapts the rendering distribution based on network conditions. When network bandwidth is limited, more rendering operations are performed locally on the client device, particularly for HUD elements that require high quality but are insensitive to network variations, thus maintaining frame quality consistency under varying network conditions.
Data Source
AI summary
Described herein is a network and renderer aware split rendering technique that enables a cloud gaming service to split execution of cloud gaming workloads between a graphics processor of a cloud gaming server and a graphics processor of a client of the cloud gaming service. The split rendering technique enables portions of a frame that most vulnerable to quality degradation at low bitrates to be rendered on the client, which results in an improvement of the quality of the final frame that is presented at the client device.