Tile-Based Remote Display Rendering for Low-Latency 3D Gaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote display rendering systems face latency issues due to network limitations, particularly in applications like 3D games, where low-latency input-action timing is critical, and excessive network latency impairs the user experience by transmitting game scene images frame-by-frame.
Innovation Solution
The system employs a tile-based rendering and transmission approach, where a remote server renders and encodes image data in a tiled format, allowing for pipelined and parallel processing, and transmission of encoded tiles over the network, enabling efficient communication and display of multiple tiles simultaneously.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If frame-by-frame image transmission is used, then network simplicity is maintained, but latency increases excessively
Solution Approach 1:
The patent divides the complete image frame into multiple independent tiles (e.g., 4-16 smaller image segments). Each tile can be rendered, encoded, and transmitted separately. This segmentation allows the system to transmit only portions of images simultaneously, reducing the time needed to display updated regions after user input, thereby lowering latency without requiring complete frame transmission.
Solution Approach 2:
The patent implements preliminary rendering and encoding of multiple image tiles in advance before user input occurs. The system pre-renders several frames of tiles and maintains them in a buffer, so when user input happens, the system can immediately transmit pre-prepared tiles without waiting for real-time rendering of the complete frame, significantly reducing perceived latency.
2Productivity
If tile-based rendering is used, then latency is reduced, but system complexity increases
Solution Approach 1:
The rendering system is segmented into independent tile rendering units that can operate in parallel. Each tile is rendered independently using the same rendering pipeline, allowing simultaneous processing of multiple tiles. This segmentation increases productivity by enabling parallel rendering while managing complexity through modular, reusable rendering code blocks.
Solution Approach 2:
The system renders and prepares more tiles than immediately needed (excessive action), maintaining a buffer of pre-rendered tiles. This ensures that when user input occurs, the system can immediately transmit available tiles without waiting for rendering to complete, maximizing rendering efficiency and minimizing latency. The excessive pre-rendering is justified by the low latency requirement in real-time applications.
3Speed
If multiple tiles are transmitted simultaneously, then display speed improves, but network bandwidth requirements increase
Solution Approach 1:
The patent applies different quality levels to different tiles based on their importance and visual impact. Critical tiles (e.g., areas with frequent changes or user focus) are transmitted with higher quality and priority, while less critical tiles use lower quality compression. This local quality differentiation allows the system to transmit multiple tiles simultaneously without uniformly increasing bandwidth consumption, optimizing the balance between display speed and data volume.
Solution Approach 2:
The system performs preliminary compression and optimization of tiles during the rendering phase, preparing them for efficient transmission. By pre-processing tiles with appropriate compression levels and prioritizing critical regions, the system reduces the actual data transmission volume while maintaining the ability to transmit multiple tiles simultaneously at high speed.
Data Source
AI summary
Apparatus, systems and methods for low latency remote display rendering using tile-based rendering systems are disclosed. In one implementation, a system includes a network interface and a content source coupled to the network interface. The content source being capable of rendering at least one tile of a tiled image, encoding the at least one tile, and providing the at least one encoded tile to the network interface before all tiles of the tiled image are rendered.