Multi-pass frame rate control using shared reference frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud-based gaming platforms face challenges in maintaining low-latency and video quality due to bandwidth constraints, where encoding game frames at incorrect sizes can lead to packet losses and delays, necessitating a method to accurately allocate bits to match available network bandwidth.
Innovation Solution
A multi-pass encoding technique using shared reference frames, where the first pass encoding estimates complexity and determines a quantization parameter, and the second pass encoding adjusts based on this parameter to generate encoded frames close to a target bitrate, minimizing the risk of overshooting and packet loss.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If single-pass encoding is used to reduce processing time, then encoding speed is improved, but bitrate accuracy deteriorates causing packet loss and delays
Solution Approach 1:
The encoding process is divided into multiple passes (e.g., analysis pass and synthesis pass) where each pass performs specific functions. The analysis pass estimates complexity and determines target bitrates, while the synthesis pass performs actual encoding using these parameters, separating complexity analysis from encoding execution to improve both speed and accuracy.
Solution Approach 2:
The system performs preliminary analysis of frame complexity and estimates target bitrates before actual encoding occurs. This preliminary action allows the encoder to prepare encoding parameters in advance, ensuring accurate bitrate control during the actual encoding process without sacrificing speed.
2Manufacturing precision
If encoding parameters are adjusted to match target bitrate, then bitrate accuracy is improved, but processing complexity increases
Solution Approach 1:
The system performs partial analysis at full resolution to estimate complexity, then uses these estimates to guide the full encoding process. This partial action approach provides sufficient accuracy for bitrate control without requiring complete analysis at all stages, reducing overall processing complexity while maintaining precision.
Solution Approach 2:
The patent replaces complex real-time bitrate adjustment mechanisms with a simpler system that uses pre-calculated complexity estimates and target bitrate parameters. This substitution simplifies the encoding process by using lookup tables and pre-determined parameters rather than complex real-time calculations.
3Manufacturing precision
If multi-pass encoding is used to improve bitrate accuracy, then bitrate control is improved, but encoding time increases
Solution Approach 1:
The system applies different processing qualities to different parts of the encoding process. The analysis pass uses lower resolution or simplified models for complexity estimation, while the synthesis pass uses full resolution with accurate parameters. This local quality differentiation reduces overall processing time while maintaining bitrate control accuracy where it matters most.
Data Source
AI summary
A multi-pass encoding operation using a shared reference frame is implemented to encode one or more gaming frames into a game stream. The multi-pass encoding operation includes producing a shared reference frame based upon a second-pass reference frame used by a second pass encoding of the multi-pass encoding operation. The encoding operation also includes performing a first encoding pass on a current frame using the shared reference frame. As a result of the first encoding pass, an estimated complexity for the current frame is determined. A second pass encoding is then performed on the current frame according to the second-pass reference frame and the estimated complexity, resulting in an encoded frame. This encoded frame is then transmitted as part of a stream to a client system.