Super-Resolution Video Coding With Low-Resolution Motion Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in efficiently managing memory consumption and computational complexity, particularly in video compression systems, due to the need for motion vector coding and motion compensation between pictures with different spatial resolutions, and the inefficiencies in coding residual information.
Innovation Solution
Incorporating a neural network approach for super-resolution restoration with residual frame coding, allowing for encoding and decoding operations to be performed at different spatial resolutions, and utilizing a multi-scale machine learning model to enhance video coding efficiency and reduce computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video coding is performed at high spatial resolution, then video quality is improved, but memory consumption and computational complexity increase
Solution Approach 1:
The video coding process is segmented into two distinct stages: low-resolution motion compensation and high-resolution residual coding. This segmentation allows the computationally intensive motion estimation to be performed at low resolution while maintaining final output quality at high resolution, thereby reducing overall computational complexity without sacrificing video quality
Solution Approach 2:
The patent introduces a resolution dimension transition by performing motion compensation at low resolution and then refining the residual at high resolution. This dimensional transition in processing resolution allows the system to avoid the computational burden of full high-resolution motion estimation while still achieving high-quality output
2Adaptability or versatility
If motion compensation is performed between pictures with different spatial resolutions, then coding flexibility is improved, but memory consumption increases
Solution Approach 1:
Different resolution qualities are applied to different stages of the coding process: low resolution for motion compensation operations and high resolution for residual coding. This local quality differentiation allows the system to use only the necessary resolution for each specific operation, reducing overall memory consumption while maintaining coding flexibility
Solution Approach 2:
The patent implements dynamic resolution switching where the processing resolution changes based on the coding stage. The system dynamically transitions from low-resolution processing during motion compensation to high-resolution processing during residual coding, optimizing memory usage at each stage rather than maintaining high resolution throughout
3Measurement precision
If residual information is coded at high resolution, then coding accuracy is improved, but computational complexity increases
Solution Approach 1:
The coding process is segmented such that only the residual information (the difference between predicted and actual values) is processed at high resolution, while the bulk motion compensation operations occur at low resolution. This segmentation ensures high coding accuracy for the critical residual portion without applying high-resolution processing to all data
Data Source
AI summary
The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for video coding using super-resolution restoration with residual frame coding. According to some examples, a computer-implemented method includes receiving a coded frame of a video; performing a video coding on the coded frame of the video to generate a resultant for the coded frame at a second lower resolution than a first resolution; upsampling the resultant in at least a vertical direction to a higher resolution than the second lower resolution to generate an upsampled resultant; generating a decoded frame based on at least the upsampled resultant; and transmitting the decoded frame to a frame buffer or to a display device.