Frame Interpolation for Streaming Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional media streaming technologies face issues with stutters and presentation anomalies due to changing bandwidth, corrupt or missing frames, and late delivery of video content, leading to an undesirable user experience.
Innovation Solution
The implementation of frame interpolation techniques, which generate interpolated frames using independent and dependent frames, thumbnails, and differential independent frames to fill in gaps and maintain smooth video playback, reducing the incidence of stutters and dropouts by extrapolating frames based on previously received data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional media streaming is used, then bandwidth efficiency is maintained, but stutters and presentation anomalies occur due to missing or corrupt frames
Solution Approach 1:
The system performs preliminary actions by delivering independent frames and thumbnails in advance before they are actually needed for playback. This allows the media device to have backup data available to generate interpolated frames when original frames are missing or corrupt, thereby maintaining playback continuity without stutters.
Solution Approach 2:
The patent introduces thumbnails as an intermediary element between the original video frames and the final playback. These thumbnails serve as a mediator that can be used to generate interpolated frames when original frames are unavailable, bridging the gap caused by missing or corrupt data.
2Reliability
If frame interpolation is implemented, then playback continuity is improved, but device complexity increases due to additional processing requirements
Solution Approach 1:
The frame interpolation process is segmented into distinct functional modules: a frame determination module that identifies missing frames, a frame generation module that creates interpolated frames using thumbnails and independent frames, and a presentation module that displays the reconstructed video sequence. This segmentation makes the complex process more manageable and implementable.
Solution Approach 2:
The system creates copies of video data in the form of thumbnails and independent frames that can be used to generate interpolated frames. Instead of requiring the original complete video stream, the system uses these copied representations to reconstruct missing portions, reducing the need for complex real-time processing of the original high-quality data.
3Reliability
If independent frames are delivered in advance, then frame gaps can be filled, but bandwidth consumption increases
Solution Approach 1:
The system applies local quality by delivering data at different quality levels for different purposes. Full-quality independent frames are delivered in advance for critical reconstruction points, while lower-quality thumbnails are used for intermediate frame generation. This approach ensures frame availability where most needed while minimizing overall bandwidth consumption.
Solution Approach 2:
The patent implements partial action by selectively delivering independent frames and thumbnails only at specific intervals rather than continuously. This partial delivery strategy provides sufficient data for frame interpolation while avoiding the excessive bandwidth consumption that would result from delivering complete high-quality video data throughout the entire streaming session.
Data Source
AI summary
When streaming content such as movies, music, or audiobooks for presentation, changes in conditions or unavailable frames may result in stutters or other negative experiences during presentation. Described herein are techniques and systems for extrapolating one or more frames based on previously received data. When an unavailable frame is determined, an interpolated frame is generated for presentation in place of the unavailable frame. The interpolated frame may be generated based on other frames that were received out of sequence, such as frames for presentation subsequent to the unavailable frame, as well as frames for presentation prior to the unavailable frame.