Video Quality Improvement Using Structure-Detail Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video quality improvement techniques face challenges in accurately aligning frames with blur, leading to deterioration in image quality improvement performance due to high computation requirements.
Innovation Solution
A video quality improvement method and apparatus based on machine learning that uses a structure-detail separation-based learning structure to stack multi-task units, allowing for harmonious learning of image quality improvement and compensation tasks with a low computation amount, effectively integrating surrounding frames for progressive image quality improvement and compensation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If progressive image quality improvement-compensation technique is used to address frame alignment issues, then image quality improvement performance is maintained, but the amount of computation required increases significantly
Solution Approach 1:
The patent divides the video processing task into separate frame processing operations, where each frame is processed independently through the convolutional neural network rather than requiring progressive compensation across multiple frames. This segmentation reduces the computational burden while maintaining deblurring performance.
2Reliability
If existing video quality improvement techniques are used, then surrounding frames are utilized for improvement, but frame alignment becomes inaccurate leading to deterioration in performance
Solution Approach 1:
The convolutional neural network performs self-service by automatically learning and performing frame alignment through convolution operations, eliminating the need for separate, error-prone alignment steps. The network inherently handles the alignment task as part of its deblurring process, improving both accuracy and efficiency.
3Manufacturing precision
If multi-task units with structure-detail separation are stacked, then deblurring performance is significantly improved, but network complexity increases
Solution Approach 1:
The patent merges structure extraction and detail enhancement tasks into a unified stacked multi-task unit architecture. By combining these functions in an integrated network structure with shared parameters, the system achieves high deblurring performance while controlling overall complexity through parameter sharing and modular design.
Data Source
AI summary
A video quality improvement method may comprise: inputting a structure feature map converted from current target frame by first convolution layer to first multi-task unit and second multi-task unit, which is connected to an output side of first multi-task unit, among the plurality of multi-task units; inputting a main input obtained by adding the structure feature map to a feature space, which is converted by second convolution layer from those obtained by concatenating, in channel dimension, a previous target frame and a correction frame of the previous frame to first multi-task unit; and inputting current target frame to Nth multi-task unit connected to an end of output side of second multi-task unit, wherein Nth multi-task unit outputs a correction frame of current target frame, and machine learning of the video quality improvement model is performed using an objective function calculated through the correction frame of current target frame.