Parallel Convolution Processing Unit for Neural Network Memory Access Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for decomposing layers or adding compressed layers in neural networks lead to an increase in the number of accesses to external memory, particularly when dealing with large intermediate feature amounts, which hampers the speed of convolution processing.
Innovation Solution
A data processing apparatus that employs parallel convolution operations using M×M and N×N data processing units, with results stored in output buffers before being transferred to external memory, reducing the need for frequent memory access by decomposing layers using Singular Value Decomposition (SVD) and switching between decomposition methods based on recognition accuracy and channel counts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If layers are decomposed or compressed layers are added in existing networks, then the amount of computation is reduced, but the number of accesses to external memory increases
Solution Approach 1:
The patent divides the convolutional layer into multiple sub-layers (first convolutional sub-layer with M×M filter, second convolutional sub-layer with N×N filter, third convolutional sub-layer with P×P filter) to process intermediate feature amounts in segments. This segmentation allows processing of large feature maps without requiring complete loading into external memory, thereby reducing memory access frequency while maintaining computational throughput.
Solution Approach 2:
The patent introduces a new dimension of processing by performing convolutions with different filter sizes (M×M, N×N, P×P) on the same input feature map. This multi-scale approach processes information at different granularities simultaneously, reducing the need for repeated memory accesses to the same data and improving computational efficiency.
2Quantity of substance
If intermediate feature amounts are large, then more data can be processed, but the number of accesses to external memory increases
Solution Approach 1:
The patent segments the processing of large intermediate feature amounts by dividing the convolutional operation into multiple sub-layers with different filter sizes. Each sub-layer processes a portion of the feature map computation, allowing large datasets to be handled in smaller computational steps without requiring complete data transfer to external memory, thus reducing access frequency.
Solution Approach 2:
The patent applies partial action by using multiple convolutional sub-layers with different filter sizes (M×M, N×N, P×P) to process the input feature map. Instead of performing a single comprehensive convolution that would require complete memory access, the computation is distributed across multiple partial operations, each accessing memory fewer times.
3Productivity
If decomposition methods are switched based on recognition accuracy and channel counts, then processing efficiency is optimized, but device complexity increases
Solution Approach 1:
The patent implements dynamic adaptation by switching between different decomposition methods based on real-time conditions such as recognition accuracy requirements and channel counts. The system dynamically adjusts the configuration of convolutional sub-layers (filter sizes M×M, N×N, P×P) to optimize processing efficiency for different input scenarios, making the architecture flexible and adaptive rather than static.
Solution Approach 2:
The patent changes key parameters (filter sizes M×M, N×N, P×P and the number of sub-layers) based on input conditions such as channel counts and accuracy requirements. By dynamically adjusting these parameters, the system optimizes the balance between computational efficiency and memory access, adapting the processing pipeline to match the specific characteristics of the input data.
Data Source
AI summary
In a data processing apparatus, an M×M data processing unit performs M×M convolution processing using data from an input buffer unit. An N×N data processing unit performs N×N convolution processing using the data from the input buffer unit. A first output buffer unit stores one of results of processing by the M×M data processing unit and the N×N data processing unit, and outputs the same to the input buffer unit. A second output buffer unit stores the other of the results of processing by the M×M data processing unit and the N×N data processing unit. The second output buffer unit transfers the result of processing to the external memory.