Implicit Neural Video Compression for Multi-Frame Redundancy Removal
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video compression methods are limited by their inability to utilize frames beyond adjacent frames, leading to performance bottlenecks and the need for improved video compression efficiency and bitrate reduction.
Innovation Solution
A video processing method that encodes an original video sequence into network model parameters of an implicit neural network, iteratively optimizing these parameters using loss functions based on spatially embedded features, frequency domain, geometric information, long-term and short-term temporal features, and rate-distortion to enhance compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional hybrid video coding frameworks or deep learning-based video coding frameworks are used to remove spatial and temporal redundancies, then video compression performance is improved to some extent, but the compression efficiency and bitrate reduction are limited due to the inability to utilize frames beyond adjacent frames
Solution Approach 1:
The patent transitions from traditional block-to-block or frame-to-frame methods to a pixel-to-pixel mapping approach across multiple frames. This dimensional change allows the system to establish correspondences between specific pixels in the current frame and pixels in reference frames, enabling effective utilization of temporal redundancy from multiple frames rather than being limited to adjacent frames only.
Solution Approach 2:
The patent introduces an implicit neural network model as an intermediary that learns the mapping relationships between pixels in the current frame and reference frames. This neural network acts as a mediator that captures complex temporal dependencies and enables efficient prediction by learning optimal mapping strategies from training data, thereby improving compression efficiency beyond traditional methods.
2Device complexity
If adjacent frames or limited number of frames are used to remove inter-frame redundancy, then computational complexity is reduced, but performance bottlenecks occur and bitrate reduction is insufficient
Solution Approach 1:
The patent performs preliminary action by pre-training the implicit neural network model on a large number of video frames to learn optimal pixel mapping relationships. This pre-training phase allows the model to capture temporal dependencies and patterns in advance, so that during actual video compression, the model can efficiently apply learned mappings without requiring complex real-time computations, thus reducing computational complexity while maintaining high temporal redundancy removal effectiveness.
Solution Approach 2:
The implicit neural network model is designed to be self-adaptive, automatically learning and optimizing the mapping relationships between frames based on the content it processes. The model adjusts its parameters through iterative optimization using loss functions that measure prediction accuracy, enabling it to adapt to different video content characteristics without requiring manual configuration or complex external control mechanisms.
Data Source
AI summary
A method, apparatus, electronic device, and computer-readable storage medium for video processing. The video processing method comprises encoding an original video sequence into network model parameters of an implicit neural network model, and iteratively optimizing the network model parameters based on the original video sequence until a predetermined requirement is met. The video processing improves rate-distortion performance, optimizes storage space and transmission bandwidth, overcomes the limitations of existing methods that can only utilize adjacent frames or limited frame information to remove inter-frame redundancy, breaks through the bottleneck of existing video compression algorithms, enhances video compression efficiency, and reduces bit rate.