Interactive Graphics Streaming via Client GPU Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current interactive graphics streaming systems require powerful servers for compute and graphics-intensive applications, leading to high power usage and cooling costs, and introduce image fidelity loss due to 'lossy' video standards like h.264, while underutilizing local GPUs in client devices and inefficiently managing bandwidth.
Innovation Solution
A method and system that execute applications on a server and render graphics on a local GPU at the client device, using semantic-driven compression to synchronize only necessary graphics data, hashing vertex buffers, and adapting to client device capabilities, allowing user input feedback and efficient bandwidth use.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If graphics applications are executed on a server and rendered using a specialist GPU, then the application can be streamed to multiple users simultaneously, but the server requires extremely powerful hardware leading to high power usage and cooling costs
Solution Approach 1:
The invention segments the graphics processing workload by separating the application execution (on server) from the graphics rendering (on client device). Instead of concentrating all compute and graphics processing on the server, the rendering task is divided and executed locally on the client's GPU, reducing the server's power consumption while maintaining the ability to serve multiple users simultaneously.
Solution Approach 2:
The client device's GPU is utilized to perform graphics rendering independently, without requiring a specialist GPU on the server. The server sends graphics instructions and data to the client, which then self-services the rendering task using its own hardware resources, eliminating the need for expensive server-side graphics hardware.
2Quantity of substance
If traditional video compression standards like h.264 are used to stream graphics from server to client, then the bandwidth requirements are reduced, but image fidelity is lost due to compression artefacts
Solution Approach 1:
Instead of compressing the final rendered video output (which causes lossy compression artefacts), the invention inverts the approach by transmitting uncompressed or losslessly compressed graphics instructions and data to the client. The rendering happens locally, and only the essential graphics data needs to be transmitted, not the final pixel stream.
Solution Approach 2:
The invention replaces the mechanical video compression system (h.264 encoding/decoding) with a graphics instruction transmission system. Instead of compressing and decompressing video frames, the system transmits graphics commands and data that are rendered locally, avoiding the lossy compression process entirely.
3Quantity of substance
If real-time video compression is performed on the server to reduce stream bandwidth, then the bandwidth requirements are lowered, but the compute intensive compression process increases server load and latency
Solution Approach 1:
The invention extracts the compression task from the real-time streaming path. Instead of compressing video in real-time on the server, the system transmits graphics instructions and data that require minimal compression, and the client performs rendering locally. This removes the compute-intensive compression step from the critical path, reducing server load and latency.
4Manufacturing precision
If high resolution graphics are streamed to support increasing display resolutions like 4K, then the image quality is maintained, but the bandwidth requirements of the encoded video stream increase significantly
Solution Approach 1:
Instead of streaming high-resolution video frames that require enormous bandwidth, the invention inverts the approach by transmitting graphics instructions and data at the native resolution and having the client's GPU render the graphics locally at the full display resolution. This maintains image quality while dramatically reducing bandwidth requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a method of streaming interactive computer graphics from a server to a client device. The method includes the steps of: intercepting graphics instructions transmitted from an application destined for a graphical processing unit (GPU) at the server; processing the graphics instructions to generate graphics data at the server; generating index information for, at least, some of the graphics data at the server; transmitting the index information in place of the graphics data to a client device; extracting corresponding graphics data stored at the client device utilizing the index information; and rendering computer graphics at a graphical processing unit (GPU) at the client device using the corresponding graphics data. A system for streaming interactive computer graphics is also disclosed.