Low-Complex Global Attention Generator for Tile-Based Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex deep learning networks require high computational resources and power consumption, making them unsuitable for electronic devices with limited resources, and integrating global attention into hardware accelerators is challenging due to memory and quality constraints.
Innovation Solution
A low-complex neural network is implemented in an electronic device with a hardware accelerator for memory-efficient computation of global attention, allowing tile-based processing of full-size images without compromising quality, and a switchable network architecture to turn global attention on/off based on use case requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex deep learning networks are used for image/video processing, then processing quality and performance are improved, but computational resource consumption and power usage increase significantly
Solution Approach 1:
The patent divides the full image into multiple tiles that can be processed independently in parallel, reducing the computational burden on each processing unit while maintaining overall processing quality through aggregation of tile-level results
2Measurement precision
If global attention mechanism is integrated into hardware accelerator, then processing quality is improved, but memory requirements and device complexity increase
Solution Approach 1:
The global attention computation is segmented into tile-level operations where each tile processes local regions independently, then combines results to achieve global attention effects without requiring the hardware accelerator to handle the entire image at once, thus reducing memory and complexity requirements
Solution Approach 2:
The patent applies approximate global attention by computing attention for each tile separately rather than computing exact global attention across the entire image, achieving sufficient processing quality with reduced computational complexity and memory usage
3Use of energy by moving object
If tile-based processing is used in hardware accelerator, then power consumption and complexity are reduced, but receptive field is limited and processing quality deteriorates
Solution Approach 1:
The patent merges tile-level processing results to reconstruct global attention maps, combining local processing efficiency with global contextual understanding, thereby maintaining processing quality while benefiting from reduced power consumption and complexity of tile-based operations
Data Source
AI summary
An electronic device for determining global attention in a deep learning model is provided. The electronic device includes a hardware accelerator, a low-complex global attention generator, a parallel switch, and a series switch. The hardware accelerator is configured to process each tile of a full-frame image and the low complex global attention generator is configured to generate a channel attention map of the full-frame image. The parallel switch is configured to bypass a connection of the channel attention map with the hardware accelerator and a series switch, configured to gate the connection of the channel attention map with the hardware accelerator.