Low Latency Game Engine Frame Broadcasting via Direct Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing live video streaming technologies often introduce significant latency due to the process of recording and broadcasting video segments, which results in delayed broadcasts and reduced frame rates, and previous solutions either offer limited latency reduction or increase processing overhead.
Innovation Solution
A system that includes a frame extractor and a broadcasting agent to directly access and broadcast raw frame data from a game engine's rendering pipeline, eliminating the need for disk storage and format conversions, thereby achieving low-latency and high-frame-rate video streaming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If video segments are recorded to disk and then broadcast, then the broadcasting process is simplified, but significant latency (5-10 seconds) is introduced
Solution Approach 1:
The patent extracts the video data directly from the game engine's rendering pipeline before it would normally be displayed to the player. By accessing the frame buffer or rendering output directly through APIs like DirectX or OpenGL, the system bypasses the traditional disk recording step, eliminating the 5-10 second latency while maintaining a streamlined broadcasting process.
Solution Approach 2:
The patent introduces an intermediary component (the frame capture module) that sits between the game engine's rendering pipeline and the broadcasting system. This intermediary directly captures frame data in memory and passes it to the broadcasting module, creating a direct data pathway that eliminates disk I/O operations and reduces latency significantly.
2Loss of time
If screenshots are captured and cached as images, then broadcast latency is reduced, but processing overhead increases and frame rate decreases
Solution Approach 1:
The patent implements continuous frame capture from the rendering pipeline at the game engine's native frame rate (e.g., 60 FPS). By continuously accessing the frame buffer in memory and passing frames directly to the broadcasting module, the system maintains high frame rates without the intermittent processing overhead of screenshot capture and caching operations.
Solution Approach 2:
The patent creates a direct memory copy of the rendered frame data from the game engine's frame buffer to the broadcasting module's buffer. This memory-to-memory copying operation is extremely fast and occurs at the full rendering frame rate, avoiding the need to write to disk or cache images, thus maintaining high productivity while achieving low latency.
3Loss of time
If frame data is accessed directly from the rendering pipeline, then latency is reduced and frame rate is maintained, but the frame data must be rendered to a native window first
Solution Approach 1:
The patent makes the game engine's rendering pipeline serve dual purposes: rendering frames to the player's display window and simultaneously providing frame data to the broadcasting module. By accessing the same rendering output through multiple pathways (one to the display, one to the broadcaster), the system achieves low latency without requiring separate rendering operations, thus avoiding additional complexity.
Data Source
AI summary
A system for low latency broadcast of animation frames includes a frame extractor stored in memory and executable to access frame data generated by a rendering pipeline of a frame generation engine. During runtime of the frame generation engine, the frame extractor exports the frame data for use external to the frame generation engine.