Neural Video Compression Without Flow-Based Warping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural video compression methods utilizing flow-based warping are not efficient for mobile devices with limited resources.
Innovation Solution
Implementing a fully convolutional neural network with auto-encoders for motion compensation, replacing flow-based warping, to extract and encode correlation features between video frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If flow-based warping is used for motion compensation in neural video compression, then motion compensation accuracy is improved, but computational complexity increases making it unsuitable for mobile devices
Solution Approach 1:
The patent replaces the mechanical flow-based warping system with a neural network-based motion compensation system. Specifically, it uses a fully convolutional neural network (FCN) to perform motion compensation without requiring explicit optical flow estimation, thereby substituting a computationally intensive mechanical process with a more efficient neural network approach that maintains accuracy while reducing complexity for mobile devices
Solution Approach 2:
The patent extracts and transmits only the essential correlation features between reference and current frames rather than transmitting full frame data or complex flow fields. By using the FCN to extract key motion correlation information and transmitting only these compressed features, the system achieves efficient motion compensation with reduced computational burden on mobile devices
Data Source
AI summary
A processor-implemented method for video compression using an artificial neural network (ANN) includes receiving a video via the ANN. The ANN extracts a first set of features of a current frame of the video and a second set of features of a reference frame of the video. The ANN determines an estimate of correlation features between the first set of features of the current frame and the second set of features of the reference frame. The estimate of the correlation features are encoded and transmitted to a receiver.