Lightweight Dense Residual Network for Mobile Video Super-Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural network-based video super-resolution methods are computationally complex, making them impractical for real-time execution on mobile devices with limited power, such as smartphones, despite advancements in hardware and frameworks like Tensorflow-lite and Pytorch Mobile.
Innovation Solution
A lightweight dense residual network (LiDeR) is designed for video super-resolution, comprising DenseRes blocks with convolution and ReLU operations, and a pixel shuffle layer for upscaling, capable of processing multiple frames concurrently and running in real-time on mobile devices without hardware-specific optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural network-based video super-resolution methods are used, then super-resolution performance is improved, but computational complexity increases making them impractical for mobile devices
Solution Approach 1:
The network is divided into multiple DenseRes blocks, each containing multiple layers with convolution and ReLU operations. This segmentation allows the complex super-resolution task to be broken down into manageable modules that can be efficiently executed on mobile devices while maintaining high resolution output quality.
Solution Approach 2:
The architecture implements nested DenseRes blocks where each block contains multiple layers, and each layer contains convolution and ReLU operations. This nested structure enables the network to maintain high representational power for superior resolution performance while organizing computations in a hierarchy that optimizes mobile device execution efficiency.
2Measurement precision
If deep neural network-based video super-resolution methods are used, then super-resolution performance is improved, but execution speed decreases making real-time processing difficult
Solution Approach 1:
The network performs preliminary feature extraction and processing in the DenseRes blocks before the final upscaling operation. By preparing feature representations in advance through multiple convolution and ReLU layers, the network achieves high resolution output while optimizing the computational path for faster execution on mobile devices.
Solution Approach 2:
The architecture transitions from processing individual frames to processing multiple frames concurrently through the DenseRes blocks. This dimensional change in the temporal domain enables parallel processing of frame sequences, significantly improving execution speed while maintaining superior resolution performance across all frames.
3Quantity of substance
If conventional super-resolution methods are used, then bandwidth requirements are reduced, but video quality deteriorates due to artifacts
Solution Approach 1:
The network creates multiple feature map copies through the DenseRes blocks, where each block receives and processes multiple versions of input features. This copying mechanism enables the network to maintain low bandwidth requirements by processing compressed feature representations while generating high-quality output through repeated feature refinement across multiple nested layers.
Data Source
AI summary
The technology described herein relates to a lightweight dense residual network for video super-resolution on mobile devices. A method for implementing a lightweight dense residual network to achieve super-resolution performance may include generating feature maps using a network based on an input of frames at a lower resolution, the network comprised of DenseRes blocks and an additional convolution operation, each DenseRes block comprising multiple layers of convolution operations and rectified linear activation function (ReLU) operations and a 1×1 convolution operation. Said feature maps are upsampled by a pixel shuffle layer in the network and the frames are output at a higher resolution, the higher resolution relative to the lower resolution by an upscaling factor.