Neural Network Pooling Kernel Decomposition for Memory Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As neural networks process complex input data, they face inefficiencies due to excessive calculations and memory access frequency, leading to performance issues in miniaturization and commercialization, particularly in deep neural networks with complex architectures.
Innovation Solution
The method involves decomposing an original pooling kernel into 1-dimensional sub-pooling kernels, performing pooling operations on input feature maps using these sub-kernels, and storing intermediate results in a share line buffer for post-processing to obtain final pooling results, reducing redundant calculations and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pooling operations are performed using original pooling kernels on input feature maps, then accurate pooling results are obtained, but computational overhead and memory access frequency increase excessively
Solution Approach 1:
The original pooling kernel is decomposed into multiple sub-pooling kernels (e.g., row kernels and column kernels). Instead of applying one large kernel, the operation is segmented into sequential applications of smaller sub-kernels, reducing memory access frequency and computational overhead while maintaining the same pooling effect.
Solution Approach 2:
Intermediate pooling results are stored in a share line buffer after each sub-pooling operation. This preliminary storage allows subsequent sub-pooling kernels to reuse these results without重新 accessing the original input feature map, reducing redundant memory accesses and improving computational efficiency.
2Reliability
If pooling operations are performed using original pooling kernels on input feature maps, then complete pooling coverage is achieved, but memory access frequency becomes excessive
Solution Approach 1:
The share line buffer stores intermediate pooling results from previous sub-pooling operations. When subsequent sub-pooling kernels need to process overlapping regions, they can retrieve these pre-computed results from the buffer instead of重新 accessing memory, significantly reducing memory access frequency and associated energy consumption.
Solution Approach 2:
Multiple sub-pooling operations are merged into a single pooling pass by sequentially applying sub-kernels and storing intermediate results in the share line buffer. This merging approach ensures complete pooling coverage while minimizing redundant memory accesses through result reuse.
3Productivity
If sub-pooling kernels are used for pooling operations, then computational overhead is reduced, but device complexity increases
Solution Approach 1:
The pooling operation is segmented into multiple sequential steps using sub-pooling kernels. While this increases the number of operations, each sub-kernel is simpler and can be implemented more efficiently in hardware, with intermediate results cached in the share line buffer to offset the increased operational steps.
Solution Approach 2:
The system dynamically manages the share line buffer to store and retrieve intermediate pooling results based on the current processing stage. This dynamic memory management optimizes the trade-off between increased operational steps and memory access efficiency, maintaining high throughput while handling the complexity of multiple sub-kernel applications.
Data Source
AI summary
A processor-implemented method of a neural network includes obtaining intermediate pooling results, respectively corresponding to sub-pooling kernels obtained by decomposing an original pooling kernel, by performing a pooling operation on input pixels included in a current window in an input feature map with the sub-pooling kernels, obtaining a final pooling result corresponding to the current window by post-processing the intermediate pooling results, and determining an output pixel value of an output feature map, based on the final pooling result, wherein the current window is determined according to the original pooling kernel having been slid, according to a raster scan order, in the input feature map.