Multi-Scale Video Enhancement for Compression Artifact Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video compression systems face challenges in efficiently removing compression artifacts and improving visual quality.
Innovation Solution
A multi-scale machine learning model is used to enhance compressed video.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional video compression is used to reduce data transmission, then bandwidth consumption is reduced, but compression artifacts and visual quality deteriorate
Solution Approach 1:
A multi-scale machine learning model is introduced as an intermediary component between the compressed video input and the final output. The model processes video at multiple resolutions (low, medium, high) and fuses features across scales to reduce compression artifacts while maintaining bandwidth efficiency. This intermediary processing stage resolves the contradiction by adding intelligence to the compression-decompression pipeline.
Solution Approach 2:
The patent applies multi-scale processing by analyzing video content at different resolution dimensions (low resolution, medium resolution, high resolution). By processing the same video data across multiple dimensional scales and fusing the results, the system achieves artifact reduction without requiring proportionally increased bandwidth, thus resolving the trade-off between compression efficiency and visual quality.
2Object-affected harmful factors
If multi-scale machine learning model is applied to enhance video quality, then visual quality is improved, but computational complexity increases
Solution Approach 1:
The machine learning model is segmented into multiple independent components that process different resolution scales (low-resolution path, medium-resolution path, high-resolution path). Each segment handles specific scale processing independently, and their results are fused together. This segmentation reduces the computational burden on any single component while maintaining overall quality improvement.
Solution Approach 2:
The system applies partial processing by selectively enhancing certain frequency components and spatial regions rather than uniformly processing all video data at maximum resolution. The multi-scale approach applies different levels of processing intensity appropriate to each scale, avoiding excessive computation while achieving sufficient quality improvement.
Data Source
AI summary
The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for training and using a multi-scale machine learning model for the enhancement of compressed video. According to some examples, a computer-implemented method includes receiving a video at a content delivery service; performing an encode on a frame of the video by the content delivery service that converts the frame from a pixel domain to a transform domain and back to the pixel domain to generate first pixel values and a first residual for a block of the frame at a first resolution; generating a first set of features, by a machine learning model of the content delivery service, for an input, at a first resolution, of the first pixel values and the first residual of the block; generating a second set of features, by the machine learning model of the content delivery service, for an input, at a second lower resolution, of second pixel values and a second residual of the block; upsampling the second set of features to the first resolution to generate an upsampled second set of features; generating a modified version of the frame based on the first set of features and the upsampled second set of features; and transmitting the modified version of the frame to a frame buffer or from the content delivery service to a viewer device.