Pruned Super-Resolution Training for Small Objects and Text
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional pruned convolutional neural networks (CNNs) struggle to learn features corresponding to small objects and textual regions due to their reduced capacity, leading to poor reconstruction of high-resolution images with noise and visual artifacts.
Innovation Solution
A framework is developed to enhance the performance of pruned CNNs by generating training images with small objects and overlaying simulated text, using a combination of static detection and YOLO-based object detection, to improve the network's ability to reconstruct realistic high-resolution images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional super-resolution networks are trained on images without occlusion, then the network structure remains complex and computationally intensive, but the network fails to handle occluded objects and produces inaccurate reconstructions
Solution Approach 1:
The network is segmented into two distinct pathways: a full-resolution pathway for processing unoccluded regions and a low-resolution pathway for processing occluded regions. This segmentation allows each pathway to be optimized for its specific function, improving overall reliability while managing complexity through specialized modular structures.
Solution Approach 2:
The network dynamically adapts its processing based on occlusion detection. When occlusion is detected in certain regions, the system automatically routes those regions through the low-resolution pathway while maintaining full-resolution processing for unoccluded areas, creating a dynamic and adaptive reconstruction system.
2Manufacturing precision
If the network processes all regions at full resolution, then reconstruction quality is maintained, but computational resources and training time increase significantly
Solution Approach 1:
Different resolution qualities are applied to different regions based on their occlusion status. Unoccluded regions receive full-resolution processing for high quality, while occluded regions are processed at lower resolution, allocating computational resources locally where they are most needed rather than uniformly across the entire image.
Solution Approach 2:
Instead of applying full-resolution processing to all regions (excessive action), the network applies partial processing at reduced resolution specifically to occluded regions where full resolution would be wasted, achieving efficient resource utilization without compromising necessary reconstruction quality.
3Reliability
If the network is designed to handle occluded objects, then reconstruction accuracy improves, but the network requires more complex data augmentation and training procedures
Solution Approach 1:
The training procedure performs preliminary occlusion detection and region classification before reconstruction processing. By pre-identifying occluded regions and preparing appropriate processing pathways in advance, the system simplifies the overall training procedure while maintaining the capability to handle occluded objects effectively.
4Productivity
If computational resources are reduced for training, then training speed increases, but the network's ability to handle diverse occlusion scenarios decreases
Solution Approach 1:
The system uses synthetic occlusion masks and simulated occluded training images to create virtual training scenarios without requiring extensive physical or real-world occluded datasets. This copying approach allows the network to learn diverse occlusion handling at reduced computational cost while maintaining adaptability to various occlusion scenarios.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An embodiment provides a method comprising detecting at least one object displayed within at least one input frame of an input video. The method further comprises cropping, from the at least one input frame, at least one cropped image including the at least one object. The method further comprises generating at least one training image by overlaying simulated text on the at least one cropped image. The method further comprises providing the at least one training image to a pruned convolutional neural network (CNN). The pruned CNN learns, from the at least one training image, to reconstruct objects and textual regions during image super-resolution.