Graphics Back Buffer Capture for Low-Latency Remote Video Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graphics device systems introduce significant encoding latencies when capturing and encoding video signals for remote rendering, degrading the user experience in latency-dependent applications like high twitch-action video games.
Innovation Solution
The system captures and encodes video signals before they reach the graphics device's external port, either by software rendering or capturing from the graphics device's back buffer, using low-latency video encoders like H.264 to reduce latency to less than approximately five milliseconds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If video signals are captured from the graphics device's external port using KVM extenders, then the system can remotely render applications, but significant encoding latencies are introduced
Solution Approach 1:
The system captures video frames from the graphics device's back buffer before they are output through the external port. This preliminary capture action occurs in advance of the traditional KVM capture point, eliminating the buffering delays that occur when capturing from the external port. The video frames are captured directly from the graphics device's internal memory (back buffer) before rendering completion, significantly reducing encoding latency while maintaining remote rendering functionality.
2Adaptability or versatility
If traditional KVM extenders are used to capture video signals, then remote display is enabled, but buffering delays degrade user experience in latency-sensitive applications
Solution Approach 1:
The invention extracts the video capture function from the external port (where buffering delays occur) and relocates it to the graphics device's back buffer (where latency is minimized). By taking out the capture operation from the traditional KVM path and implementing it directly at the graphics device's internal buffer, the system eliminates the buffering delays that degrade user experience in games and real-time applications while preserving the remote display capability.
3Ease of manufacture
If video encoding is performed after graphics device output, then system compatibility is maintained, but latency increases to unacceptable levels
Solution Approach 1:
The system performs video encoding in advance by capturing frames from the graphics device's back buffer before the traditional output stage. This preliminary encoding action occurs at the point where the video is actually generated, rather than waiting for external port output. The encoding process begins immediately when the graphics device completes rendering a frame, significantly improving video transmission efficiency while maintaining compatibility through standard encoding protocols like H.264.
Data Source
AI summary
Disclosed are various embodiments that reduce video encoding latency for remotely executed applications. An application is executed in response to a client request. A video frame generated by the application is obtained before the video frame is sent to an external port of a graphics device. The video frame is encoded into a compressed video stream. The compressed video stream is sent to the client.