2D Processing Element Array for Weight-Stationary CNN Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks, particularly convolutional neural networks, face computational intensity and energy expenditure due to repeated multiplication operations and frequent memory access during convolution operations, necessitating a hardware-efficient acceleration method.
Innovation Solution
A computation unit comprising a 2D array of processing elements that unrolls computations in one or more dimensions of the input, weight, and output tensors, employing a weight-stationary dataflow architecture to perform parallel dot-product operations, reducing data movement and optimizing hardware efficiency by reusing input data and preloading weight values in registers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional convolution operations are performed with repeated multiplication and memory access, then computational accuracy is maintained, but computational intensity and energy expenditure increase significantly
Solution Approach 1:
The computation unit is segmented into multiple processing elements (PEs) arranged in a 2D array, where each PE handles a specific portion of the convolution computation. This segmentation allows parallel processing of different regions of the input feature map, significantly reducing computation time and energy expenditure per computation while maintaining accuracy.
Solution Approach 2:
The patent transitions from sequential 1D convolution computation to parallel 2D array-based computation. By organizing PEs in a two-dimensional spatial arrangement that matches the kernel dimensions, the system performs computations across multiple dimensions simultaneously, achieving substantial speedup and energy efficiency improvements.
2Productivity
If conventional convolution operations are performed with repeated multiplication and memory access, then computational accuracy is maintained, but computational intensity increases
Solution Approach 1:
Each processing element in the 2D array is designed as a universal computational unit capable of performing the same convolution operation on different input data. This multi-functionality allows the hardware to handle various kernel sizes, stride values, and input feature maps using the same architectural blueprint, reducing overall device complexity while maintaining high throughput.
Solution Approach 2:
The patent uses identical copies of the processing element circuitry arranged in a 2D array. Each PE is a replicated instance of the same hardware module, which simplifies design verification, manufacturing, and maintenance. The copying approach enables parallel processing without requiring complex heterogeneous hardware designs.
3Loss of energy
If weight values are stored in registers and input data is reused, then hardware efficiency is optimized, but data movement is reduced
Solution Approach 1:
Weight values from the kernel are pre-loaded into registers of the processing elements before the convolution computation begins. This preliminary action eliminates the need for repeated memory accesses during the computation phase, significantly reducing power consumption. The weights remain stationary in registers throughout the computation, hence the term 'weight-stationary' architecture.
Solution Approach 2:
Each processing element serves itself by maintaining its required weight values in local registers, eliminating the need for continuous external memory access. The PEs independently perform computations using their locally stored weights and incoming input data, reducing overall system power consumption and memory bandwidth requirements.
Data Source
AI summary
A computation unit for performing a computation of a neural network layer is disclosed. A number of processing element (PE) units are arranged in an array. First input values are provided in parallel in an input dimension of the array during a first processing period, and a second input values are provided in parallel in the input dimension during a second processing period. Computations are performed by the PE units based on stored weight values. An adder coupled to the first set of PE units generates a first sum of results of the computations by the first set of PE units during the first processing cycle, and generates a second sum of results of the computations during the second processing cycle. A first accumulator coupled to the first adder stores the first sum, and further shifts the first sum to a second accumulator prior to storing the second sum.