Long Term Reference Frame Management With Error Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In video communication, when compressed video is sent over a network, lost packets can cause errors that propagate to subsequent frames, and managing reference frames across multiple decoders is challenging, especially in error-prone environments like multipoint video teleconferencing systems, where ensuring all decoders use the same long-term reference picture is crucial for error resilience.
Innovation Solution
A method and apparatus that manage the use of reference frames by sending a reference frame message to multiple decoders, receiving feedback on successful receipt, and repeating the message until all decoders have access to the indicated frame, ensuring synchronization and avoiding the use of independently coded frames upon error detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the source sends I-frames frequently to correct errors, then error resilience is improved, but compression efficiency deteriorates due to increased bit usage
Solution Approach 1:
The system uses feedback from decoders to the encoder about received frame quality and reference frame availability. This feedback enables the encoder to adaptively select reference frames and coding modes based on actual network conditions and decoder states, correcting errors efficiently without excessive I-frame transmission
Solution Approach 2:
The reference frame selection is made dynamic rather than static. The encoder can switch between different reference frames (short-term vs. long-term) based on feedback about which frames were successfully received by decoders, allowing adaptive error recovery without fixed I-frame schedules
2Productivity
If the encoder uses different reference frames for different decoders, then individual decoder performance is optimized, but synchronization between decoders deteriorates
Solution Approach 1:
The reference frame buffer is segmented into short-term reference frames (recently decoded frames) and long-term reference frames (selectively stored frames). Different decoders can use different segments based on their reception status, while maintaining overall system coherence through coordinated reference frame management
Solution Approach 2:
Long-term reference frames act as intermediaries between the encoder and multiple decoders. These frames are selectively stored in both encoder and decoder memory, serving as common reference points that mediate between different decoder states and enable synchronized error recovery
3Reliability
If the encoder sends reference frame instructions to all decoders, then error recovery capability is improved, but network overhead increases when some decoders fail to receive instructions
Solution Approach 1:
The encoder proactively sends reference frame instructions to all decoders before errors occur or when switching reference frames is beneficial. This preliminary action ensures decoders have appropriate reference frames ready, reducing the need for reactive error correction traffic and improving overall error recovery efficiency
Data Source
Figure 1A~1B
Figure 2A
Figure 2B
AI summary
An apparatus, software encoded in tangible media, and a method at an encoder. The method includes sending compressed video data including a reference frame message to create a long term reference frame to a plurality of decoders at one or more destination points, receiving feedback from the decoders indicative of whether or not the decoders successfully received the reference frame message, and in the case that the received feedback is such that at least one of the decoders did not successfully receive the reference frame message or does not have the indicated recent frame, repeating sending a reference frame message to create the long term reference frame. Using the method can replace I-frame error recovery with long term reference frames, even in the case where the reference frame management messages are lost to at least one decoder.