Game-Generated Motion Vectors for Low-Latency Remote Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding methods in real-time video streaming, particularly in remote gaming applications, are sensitive to latency due to computationally intensive block-based motion estimation, which hinders efficient encoding and quality of video transmission.
Innovation Solution
Integrating a 3D graphics engine with a video codec engine to provide pre-generated per-pixel motion vectors, converting them into per-block motion vectors, and injecting these into the encoding process, thereby skipping the motion estimation step and reducing computational power and encoding time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If block-based motion estimation is used in existing video coding methods, then video encoding quality is improved, but encoding time and computational power increase significantly
Solution Approach 1:
The graphics engine performs preliminary action by generating per-pixel motion vectors during the rendering process before video encoding begins. These motion vectors are then converted to per-block motion vectors and injected into the video encoder, eliminating the need for the encoder to perform computationally intensive block-based motion estimation. This preliminary generation of motion data resolves the contradiction by providing high-quality motion information in advance, achieving both high encoding quality and reduced encoding time.
2Manufacturing precision
If block-based motion estimation is used in existing video coding methods, then video encoding quality is improved, but computational power requirements increase
Solution Approach 1:
The graphics engine serves itself by generating per-pixel motion vectors as a byproduct of its own rendering operations. Instead of requiring a separate, computationally intensive motion estimation process in the video encoder, the system reuses motion data already computed during graphics rendering. This self-service approach eliminates redundant computational work while maintaining high encoding quality, thereby reducing overall computational power requirements.
3Device complexity
If traditional video encoding and graphics rendering operate separately, then system simplicity is maintained, but encoding time and latency increase
Solution Approach 1:
The invention merges the video encoding process with the graphics rendering process by integrating the video codec engine with the graphics engine. This integration allows the graphics engine to provide per-pixel motion vectors directly to the video encoder during rendering, eliminating the need for separate, time-consuming motion estimation. The merged system achieves reduced encoding time and latency while maintaining manageable complexity through shared data structures and coordinated operation between the two engines.
Data Source
AI summary
Systems and methods for integrated graphics rendering are disclosed. In certain embodiments, the systems and methods utilize a graphics engine, a video encoding engine, and remote client coding engine to render graphics over a network. The systems and methods involve the generation of per-pixel motion vectors, which are converted to per-block motion vectors at the graphics engine. The graphics engine injects these per-block motion vectors into a video encoding engine, such that the video encoding engine may convert those vectors into encoded video data for transmission to the remote client coding engine.