Client-Side Frame Prediction for Skipped Video Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-latency issues and packet loss during transmission of compressed video over computer networks lead to noticeable performance problems, such as freezing screens and visual artifacts, especially in low-bandwidth conditions where frames are dropped or skipped.
Innovation Solution
A method involving the transmission of both a primary and a secondary video stream, where the secondary stream contains key minimal data like motion vectors and residuals, allowing the decoder to reconstruct frames in case of packet loss, and client-side frame prediction to minimize visual impact when frames are skipped.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video frames are transmitted at full data rate to maintain video quality, then manufacturing precision (video quality) is improved, but loss of time (latency) worsens due to network transmission delays and packet loss recovery time
Solution Approach 1:
The patent pre-calculates and buffers predicted frame data at the client device before actual frames are received. When packet loss occurs, the decoder can immediately use the pre-computed predicted frame instead of waiting for retransmission or alternative reference frames, thereby reducing latency while maintaining video quality through accurate prediction algorithms
Solution Approach 2:
The patent creates predicted frame copies using motion estimation and compensation techniques. Instead of transmitting every frame at full quality, the system generates local copies of expected frames based on motion vectors and reference frames, allowing rapid substitution when original frames are lost, thus reducing transmission time while preserving visual quality
2Reliability
If packet loss protection mechanisms are implemented to maintain reliability, then reliability is improved, but device complexity worsens due to additional decoding and reconstruction processes
Solution Approach 1:
The patent extracts only the essential prediction data (motion vectors, reference frame selections) needed for frame reconstruction and transmits this minimal set separately from full frame data. This allows the decoder to build reliability through prediction without processing the full complexity of complete frame decoding, reducing overall device complexity while maintaining packet loss protection
Solution Approach 2:
The client device performs self-service frame reconstruction by using locally stored reference frames and motion compensation algorithms to generate predicted frames when packets are lost. This eliminates the need for complex server-side retransmission protocols or sophisticated network-layer error correction, reducing system-wide complexity while maintaining reliability
3Manufacturing precision
If motion vector precision is increased to improve frame prediction accuracy, then manufacturing precision (prediction accuracy) is improved, but use of energy (computational resources) worsens
Solution Approach 1:
The patent applies motion estimation at varying precision levels depending on the situation. For stable scenes, coarse motion vectors are used requiring minimal computation. For high-motion scenes where accuracy is critical, finer-grained motion estimation is applied selectively. This partial application of high-precision processing maintains prediction accuracy where needed while conserving computational energy in stable conditions
Data Source
AI summary
A method for delivering video over a network includes encoding, by a server, a sequence of video frames as a compressed video stream comprising an initial I-frame followed by a plurality P-frames, the sequence having a skipped frame which is not encoded. The compressed video stream is transmitted over a network to a client device, the skipped frame not being transmitted. In response to detecting the skipped frame, the client device generating a predicted frame that replaces the skipped frame in the compressed video stream, the predicted frame being generated based on extending delta information from one or more previous frames decoded by the client device.