Cloud Gaming Overlay Multiplexing for Thin Client Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud gaming technologies face challenges in efficiently processing and streaming high-quality video content with overlays on thin client devices, which lack the necessary hardware resources for transcoding, decoding, and re-encoding, leading to performance issues and hardware limitations.
Innovation Solution
A graphics processing system that employs a pass-through mode and look-ahead encoding with multiplexing to efficiently handle content streaming, allowing for the direct processing and streaming of video content with overlays without the need for extensive re-encoding, utilizing a graphics processing unit (GPU) to accelerate graphics and media operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional streaming software captures and re-encodes content from cloud gaming platforms, then video content can be processed with overlays, but the computational load on thin client devices increases beyond their hardware capabilities
Solution Approach 1:
The patent extracts the encoding function from the thin client device and relocates it to the cloud server. The client device only performs decoding and compositing of video frames with overlays, while the cloud server handles the computationally intensive encoding of the game stream. This separation allows thin client devices to process overlays without requiring high-capability encoding hardware.
Solution Approach 2:
The patent introduces an intermediary compositing buffer that receives both the decoded game video frames and overlay content, then combines them before sending to the display. This intermediary structure allows overlay processing to occur at the client device without requiring the device to perform full re-encoding, thus reducing hardware requirements while maintaining versatility.
2Manufacturing precision
If full re-encoding is performed on thin client devices, then video quality can be optimized, but processing time and computational resources are insufficient
Solution Approach 1:
The cloud server performs preliminary encoding of the game stream into a compressed format before transmission to the client device. This preliminary action allows the client device to receive pre-encoded video data that requires minimal processing, avoiding the time-consuming full re-encoding operation while still maintaining video quality through the use of optimized encoding parameters set on the server.
Solution Approach 2:
Instead of performing complete re-encoding on the client device, the system applies partial processing - the client device only performs decoding and compositing operations, leaving the intensive encoding work to be performed in advance by the cloud server. This partial action approach achieves sufficient video quality without the excessive processing time required for full client-side re-encoding.
3Productivity
If high throughput hardware is used on client devices for capturing and encoding, then streaming performance improves, but the cost and complexity of thin client devices increases
Solution Approach 1:
The patent extracts the high-throughput encoding function from the client device and places it on the cloud server. The client device only requires basic decoding and compositing capabilities, which can be achieved with low-power hardware. This extraction allows high streaming performance to be maintained through server-side encoding while keeping client device hardware complexity low.
Solution Approach 2:
The patent replaces the mechanical hardware encoding system on the client device with a software-based decoding and compositing system. The computationally intensive encoding operation is substituted by receiving pre-encoded streams from the cloud server, replacing the need for high-throughput encoding hardware on client devices with simpler hardware that can handle decoding and frame composition.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
An apparatus to facilitate video streaming for cloud gaming is disclosed. The apparatus includes a processor to receive a first compressed content bitstream from a first server device; capture input content to utilize as overlay content for the first compressed content bitstream; determine whether one of a first mode of pass-through encoding is enabled or a second mode of look-ahead encoding is enabled; and responsive to the first mode of pass-through encoding being enabled: generate a second content bitstream representing the input content; multiplex the first compressed content bitstream with the second content bitstream to generate a third combined content bitstream; and transmit the third combined content bitstream to a streaming server device for streaming to one or more users.