Image Resolution Assessment for Low-Quality Mobile Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) deployed on mobile devices experience significant accuracy drops due to the discrepancy between high-quality training datasets and low-quality streaming videos, primarily caused by low image resolution, which affects the activation of low-level convolutional kernels and impairs inference accuracy.
Innovation Solution
Implementing sparse windowed Fourier transforms on localized image chips with Gaussian windows to assess image resolution, using parallel processing and a 'paper folding' technique to efficiently calculate 1D energy spectrum, enabling accurate resolution assessment in milliseconds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If DNN is deployed on mobile devices with low-resolution video streams, then device compatibility and real-time processing are improved, but inference accuracy deteriorates
Solution Approach 1:
The system performs preliminary resolution assessment of video frames before DNN processing. By evaluating the resolution quality in advance, the system can determine whether to proceed with DNN inference, switch to alternative processing modes, or request higher quality input, thereby preventing accuracy degradation before it occurs.
Solution Approach 2:
The system dynamically changes processing parameters based on detected resolution quality. When low resolution is detected, it adjusts DNN configuration parameters, activation thresholds, or processing depth to match the input quality, optimizing performance for the given resolution conditions rather than using fixed parameters designed for high-quality input.
2Measurement precision
If traditional resolution assessment methods are used, then measurement completeness is improved, but processing speed deteriorates
Solution Approach 1:
Instead of analyzing the entire video frame, the system segments the image into multiple regions and selects representative samples from different areas. This segmentation approach maintains measurement precision by capturing spatial variations in resolution quality while dramatically reducing the computational load compared to full-frame analysis.
Solution Approach 2:
The system extracts only the essential frequency information needed for resolution assessment using Fourier transform, discarding redundant spatial details. By taking out only the critical spectral components that indicate resolution quality, the system achieves accurate measurement with minimal processing requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for image resolution assessment of a digital image includes extracting a plurality of image chips from the digital image according to a predetermined sampling pattern. A Gaussian window is applied to each image chip of the plurality of image chips to generate a plurality of processed image chips. Two-dimensional (2D) energy spectrum is determined for each processed image chip of the plurality of processed image chips using a discrete Fourier transformation (DFT). One-dimensional (1D) energy spectrum is determined for each processed image chip of the plurality of image chips using the 2D energy spectrum. A threshold is applied to the 1D energy spectrum to obtain an image resolution for each processed image chip of the plurality of processed image chips. A video characteristic of the digital image is adjusted based on a maximum image resolution of the image resolutions determined for the plurality of processed image chips.