Bidirectional Video Compression With Learned Reference Frame Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing bidirectional video compression methods assume uniform motion and fixed hierarchical structures, which are inadequate for complex real-world scenarios, leading to suboptimal compression performance.
Innovation Solution
Employ a Siamese neural network to generate a single reference frame from two bidirectional frames, incorporating feature similarity values, and design an inverse channel-wise entropy model considering uneven entropy distribution among channels to enhance compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed hierarchical structure is used for bidirectional video compression, then the compression process is simple and efficient, but it cannot handle complex real-world motion patterns effectively
Solution Approach 1:
The patent replaces the fixed hierarchical structure with a dynamic reference frame generation mechanism. A neural network processes two reference frames and the current frame to adaptively generate an optimal single reference frame, allowing the system to dynamically adjust to complex motion patterns while maintaining compression efficiency
Solution Approach 2:
The patent changes the parameter of reference frame generation from a fixed hierarchical approach to a learned parameter transformation. By using a neural network to compute feature similarity values and generate reference frames based on entropy distribution, the system adapts to different motion complexities without changing the fundamental compression framework
2Reliability
If two reference frames are used for bidirectional compression, then compression performance can be improved, but the complexity of processing increases
Solution Approach 1:
The patent merges the processing of two reference frames into a single unified reference frame generation process. Instead of handling two separate reference frames independently, the system uses a neural network to process both reference frames and the current frame together, generating one optimized reference frame that captures the essential information from both directions
Solution Approach 2:
The patent introduces a neural network as an intermediary component that simplifies the complexity of processing two reference frames. This intermediary learns to extract meaningful features and generate optimal reference frames automatically, reducing the manual complexity of bidirectional processing while maintaining high compression performance
3Ease of manufacture
If channels are processed independently, then the compression process is simple, but it does not account for entropy distribution across channels
Solution Approach 1:
The patent applies local quality by processing different channels with different strategies based on their entropy characteristics. Instead of uniform processing, the system identifies channels with higher entropy and applies enhanced processing to those specific channels, optimizing compression efficiency for each channel according to its properties
Solution Approach 2:
The patent segments the channel processing into two groups: high-entropy channels and low-entropy channels. This segmentation allows the system to apply different compression strategies to different channel types, with high-entropy channels receiving more attention and resources while low-entropy channels are processed more efficiently
Data Source
AI summary
A computer-implemented method for learning-based bidirectional video compression includes, given a current frame, generating a single reference frame from the current frame and bidirectional frames by using a neural network, estimating a motion between the current frame and the reference frame, obtaining a reconstructed motion by inputting the motion to a motion encoder and decoder, generating a set of temporal contexts based on the reconstructed motion and propagated feature, and compressing the current frame based on the temporal contexts by an inverse channel-wise entropy model which is adapted to reconstruct the relationship among channels such that the channels with less entropy are coded first and the channels with larger entropy are coded with the help of the previously coded channels.