Substitutional Neural Residual Compression for Video Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression methods, such as H.264/AVC, HEVC, and VVC, rely on handcrafted coding tools and lack the ability to fully leverage nonlinearity and nonlocal spatiotemporal correlations for improved compression quality.
Innovation Solution
A method and apparatus for substitutional neural residual compression using Deep Neural Networks (DNNs) to estimate motion vectors, generate predicted image frames, and encode residuals, allowing for flexible bitrate control without the need for multiple model instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If handcrafted coding tools are used in video compression, then device complexity is reduced, but compression performance is limited due to inability to leverage nonlinearity and nonlocal spatiotemporal correlations
Solution Approach 1:
The patent changes the fundamental parameters of video compression by replacing handcrafted linear tools with deep neural networks that can model nonlinear relationships. The DNNs learn optimal transformation parameters from data, enabling capture of nonlocal spatiotemporal correlations that traditional methods miss, thus improving compression performance despite increased model complexity.
Solution Approach 2:
The patent substitutes mechanical handcrafted coding tools with intelligent deep neural network systems. Instead of manually designed linear transforms and hand-tuned parameters, the system uses trained DNNs that automatically learn optimal compression representations, replacing the mechanical approach with an adaptive intelligent system.
2Adaptability or versatility
If multiple model instances are used to achieve flexible bitrate control, then bitrate flexibility is improved, but device complexity increases
Solution Approach 1:
The patent introduces dynamic adaptability through a single DNN model that can adjust its behavior based on target bitrate parameters. Instead of using multiple static model instances for different bitrates, the system uses one dynamic model that adapts its compression strategy based on the desired bitrate, reducing complexity while maintaining flexibility.
Solution Approach 2:
The patent creates a universal DNN model that can handle multiple bitrate requirements through a single instance. The model is designed to be multi-functional, capable of producing compressed representations at various bitrates by adjusting its internal parameters and transformation strategies, eliminating the need for separate specialized models for each bitrate target.
Data Source
AI summary
A method of substitutional neural residual compression is performed by at least one processor and includes estimating motion vectors, based on a current image frame and a previous reconstructed image frame, obtaining a predicted image frame, based on the estimated motion vectors and the previous reconstructed image frame, and subtracting the obtained predicted image frame from the current image frame to obtain a substitutional residual. The method further includes encoding the obtained substitutional residual, using a first neural network, to obtain an encoded representation, and compressing the encoded representation.


