CNN Memory Allocation via Data Reuse Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Designing efficient CNN processes for multi-accelerator SoCs is challenging due to high computational and memory intensity, with existing methods being time-consuming and costly, especially in minimizing external memory accesses while adhering to area and power constraints.
Innovation Solution
The Reuse Analysis and Memory Allocation Based Scheduling (RAMABS) framework determines optimal on-chip memory allocation and scheduling schemes by analyzing data reuse and external memory access costs, reducing external memory accesses and design exploration time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If CNN processes are implemented on multi-accelerator SoCs, then computational capability for object recognition is improved, but external memory access cost and energy consumption increase
Solution Approach 1:
The patent applies preliminary action by pre-loading input feature maps and kernel data into on-chip memory before processing begins. The method determines which data to preload based on the CNN layer configuration and processing requirements, storing this data in on-chip memory in advance to avoid costly external memory accesses during the actual computation phase.
Solution Approach 2:
The patent introduces on-chip memory as an intermediary between external memory and processing units. This intermediary buffer stores frequently accessed data (input feature maps and kernels) locally, allowing processing units to access data quickly without repeatedly accessing external memory, thus reducing energy consumption while maintaining computational capability.
2Use of energy by moving object
If more on-chip memory is allocated to reduce external memory accesses, then energy consumption decreases, but on-chip memory area increases
Solution Approach 1:
The patent applies local quality by selectively allocating on-chip memory to specific data types based on their access patterns and importance. Instead of uniformly allocating memory, the method identifies which input feature maps and kernel data require local storage to minimize external accesses, and allocates on-chip memory resources accordingly to each layer and data type.
Solution Approach 2:
The patent uses parameter changes by dynamically adjusting the amount of on-chip memory allocated to different CNN layers based on their specific requirements. The method analyzes layer configuration parameters (such as number of filters, input/output dimensions) to determine optimal memory allocation for each layer, balancing energy savings against available on-chip memory area.
3Measurement precision
If exhaustive search is used to find optimal scheduling schemes, then optimization accuracy is improved, but design exploration time increases significantly
Solution Approach 1:
The patent applies segmentation by dividing the CNN processing into discrete layers and identifying independent scheduling decisions for each layer. Instead of searching for optimal schedules across the entire CNN at once, the method segments the problem into layer-by-layer optimization, where each layer's scheduling can be determined independently based on its specific characteristics and data reuse opportunities.
Solution Approach 2:
The patent uses partial action by implementing a simplified optimization approach that focuses on the most critical factors (data reuse patterns, layer configuration) rather than exhaustively evaluating all possible scheduling parameters. This partial optimization achieves sufficient accuracy for practical deployment while dramatically reducing design exploration time compared to exhaustive search.
Data Source
AI summary
A method of configuring a System on Chip to execute a CNN process comprising CNN layers, the method comprising, for each schedule: determining memory access amount information describing how many memory accesses are required; expressing the memory access amount information as relationships describing reusability of data; combining the relationships with a cost of writing and reading from external memory, to form memory access information; determining a memory allocation for on-chip memory of the SoC for the input FMs and the output FMs; and determining, dependent upon the memory access information and the memory allocation for each schedule; a schedule which minimises the memory access information of external memory access for the CNN layer of the CNN process; and a memory allocation associated with the determined schedule.