Video Processing Using Residual Neural Network Parameters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video processing methods face challenges in efficiently performing super-resolution, tone mapping, frame interpolation, motion deblurring, denoising, and compression artifact removal, particularly in scenarios where neural network model parameters need to be transmitted efficiently and processed effectively across different temporal portions of a video.
Innovation Solution
A video processing method and device that utilize light-weight residual dense blocks within neural networks, configured in a cascade structure, to perform these operations by receiving and processing video frames in a manner that splits frames into spatial regions and adjusts model parameters based on content-adaptive training, minimizing residues and optimizing transmission and processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a first neural network processes the entire video, then processing coverage is complete, but transmission efficiency decreases due to large model parameter size
Solution Approach 1:
The patent segments the neural network model parameters into two parts: a first model parameter for the first neural network that processes the entire video, and second model parameters for second neural networks that process specific temporal portions. By transmitting only the differences (residues) between these model parameters, the system reduces transmission data volume while maintaining complete processing coverage across different video segments
Solution Approach 2:
The patent applies different neural network models to different temporal portions of the video based on local characteristics. The first neural network handles general processing for the entire video, while second neural networks with customized model parameters handle specific temporal portions that require specialized processing, optimizing both coverage and efficiency
2Measurement precision
If multiple second neural networks process different temporal portions, then processing precision improves, but device complexity increases
Solution Approach 1:
The patent divides the video processing task into multiple temporal portions, each handled by a second neural network with specialized model parameters. This segmentation allows each network to be optimized for specific content characteristics, improving processing precision while the modular structure helps manage device complexity through organized parameter storage and retrieval mechanisms
3Manufacturing precision
If model parameters are customized for content-adaptive training, then restoration fidelity improves, but transmission overhead increases
Solution Approach 1:
The patent extracts only the essential differences between the first model parameter and second model parameters, transmitting only these residues rather than complete model parameters. This extraction approach maintains content-adaptive customization for high restoration fidelity while significantly reducing transmission overhead by sending only the necessary differential information
Solution Approach 2:
The patent changes the parameter representation from absolute model parameters to differential residues. By encoding model parameters as changes relative to a base model, the system achieves content-adaptive training benefits with reduced transmission requirements, as the residues contain only the necessary deviation information
Data Source
AI summary
Disclosed are a video processing method and a device therefor. The video processing method may include receiving a video comprising a plurality of temporal portions, receiving a first model parameter corresponding to a first neural network to process the video entirely, receiving residues between the first model parameter and a plurality of second model parameters corresponding to a plurality of second neural networks to process the plurality of temporal portions, and performing at least one of super-resolution, reverse or inverse tone mapping, tone mapping, frame interpolation, motion deblurring, denoising, and compression artifact removal on the video based on the residues.