Neural Network Accelerator Parallel Convolution Computation and Data Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network accelerators lack parallelism in convolution computation and data loading, leading to long reasoning times and high power consumption, especially for large-scale networks, which affects real-time performance and increases chip area and cost due to large on-chip convolution kernel caches.
Innovation Solution
A parallel method for convolution computation and data loading that involves storing input feature maps in a cache, loading convolution kernels in sub-blocks, and executing convolution computations independently, with a mechanism to interchange caches and efficiently manage kernel storage to reduce computation time and chip area.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If convolution computation and data loading are executed sequentially in the prior art, then the system is simple to implement, but the reasoning time becomes too long and real-time performance deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-loading convolution kernels into cache memory before they are actually needed for computation. The system divides convolution kernels into groups and loads them in advance during idle periods, so that when computation is ready, the data is already available. This eliminates waiting time and enables parallel execution of computation and data loading operations.
Solution Approach 2:
The patent segments the convolution kernels into multiple groups (first group, second group, etc.) and divides the cache into corresponding sub-blocks. Each group of kernels is loaded into a specific cache sub-block independently. This segmentation allows different groups to be loaded and computed in parallel, improving throughput while maintaining manageable system complexity through modular organization.
2Loss of time
If large on-chip convolution kernel caches are used to ensure normal computation in the prior art, then computation can proceed without waiting for data loading, but the chip area increases and cost increases
Solution Approach 1:
The patent applies partial action by loading only the necessary portion of convolution kernels at any given time. Instead of loading all kernels into cache simultaneously, the system loads kernels in groups (first group, second group) based on computation needs. The cache size is optimized to hold only the current group being processed, reducing total cache memory requirements while maintaining continuous computation through overlapping load-compute operations.
Solution Approach 2:
The system performs preliminary loading of convolution kernel groups into cache sub-blocks before computation begins. By preparing data in advance in a staged manner, the system minimizes waiting time during actual computation without requiring the cache to hold all possible kernels simultaneously, thus reducing the required chip area.
3Duration of action of stationary object
If all convolution kernels are loaded into cache at once in the prior art, then computation can be continuous, but the data loading time and power consumption increase
Solution Approach 1:
The patent segments the convolution kernel loading process into multiple stages, with kernels divided into groups that are loaded sequentially in the background. While one group is being computed, the next group is being loaded in parallel. This segmentation distributes the loading workload over time, reducing peak power consumption during data transfer while maintaining computation continuity through overlapping operations.
Solution Approach 2:
The patent ensures continuity of useful action by overlapping computation and data loading operations. While the processor computes with the current group of kernels, the system simultaneously loads the next group into cache in the background. This continuous overlapping of useful actions (computation and loading) maintains computational throughput while spreading power consumption over time, avoiding peak demands.
Data Source
AI summary
Disclosed are a parallel method and device for convolution computation and data loading of a neural network accelerator. The method needs two input feature maps and two convolution kernel cache blocks, and sequentially stores the input feature maps and 64 convolution kernels into cache sub-blocks according to a loading length, so as to execute convolution computation and simultaneously load data of a next group of 64 convolution kernels.