Neural Network Processor Architecture for DNN Performance
Solution Overview
Problem
Deep Neural Networks (DNNs) pose significant computational challenges due to their complexity and data intensity, particularly in training and evaluation, which grows with larger datasets and more complex networks, leading to inefficiencies in both energy consumption and performance.
Innovation Solution
A scalable processor architecture with heterogeneous processing tiles and a 3-tiered interconnect topology is designed to leverage the compute and communication characteristics of DNNs, incorporating compute-intensive and memory-intensive tiles, and a low-overhead synchronization mechanism to improve processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional processors execute neural network instructions, then general-purpose computing is maintained, but processing performance and energy efficiency deteriorate due to the computational intensity and data movement requirements of DNNs
Solution Approach 1:
The processor is divided into heterogeneous processing tiles including compute-intensive tiles with multiple processing elements and memory-intensive tiles with large data arrays. This segmentation allows specialized handling of different computational workloads, improving both performance and energy efficiency for neural network operations by matching computation types to appropriate tile architectures.
Solution Approach 2:
Different regions of the processor have specialized characteristics: compute-intensive tiles are optimized for arithmetic operations with multiple processing elements, while memory-intensive tiles are optimized for data storage and access with large arrays. This local quality optimization reduces data movement between regions and improves energy efficiency for specific computational tasks.
2Reliability
If larger datasets and more complex networks are used, then model accuracy and capability improve, but computational complexity and data movement requirements increase
Solution Approach 1:
The processor introduces a spatial dimension to computation by organizing processing elements and data arrays in multi-dimensional configurations within tiles. This allows parallel processing of large datasets across multiple processing elements simultaneously, handling complex neural networks without proportionally increasing sequential computational complexity.
Solution Approach 2:
Multiple copies of processing elements and data arrays are created within and across tiles to enable parallel processing. This copying strategy allows the same computational operations to be performed simultaneously on different data partitions, improving throughput for complex models while maintaining manageable per-element complexity.
3Productivity
If data movement between memory and processor is increased, then more comprehensive computations can be performed, but energy consumption and processing time increase
Solution Approach 1:
Compute-intensive and memory-intensive tiles are merged into a unified processor architecture with direct interconnects. This merging eliminates the need for frequent data transfers to external memory, allowing comprehensive computations to be performed with data kept within the processor boundary, thereby reducing both time and energy overhead.
Solution Approach 2:
Data is pre-loaded into memory-intensive tiles before computation begins, and compute-intensive tiles are pre-configured with processing elements ready for operation. This preliminary action reduces idle time during computation and minimizes data movement overhead during the actual processing phase.
Data Source
AI summary
Methods and apparatuses relating to processing neural networks are described. In one embodiment, an apparatus to process a neural network includes a plurality of fully connected layer chips coupled by an interconnect; a plurality of convolutional layer chips each coupled by an interconnect to a respective fully connected layer chip of the plurality of fully connected layer chips and each of the plurality of fully connected layer chips and the plurality of convolutional layer chips including an interconnect to couple each of a forward propagation compute intensive tile, a back propagation compute intensive tile, and a weight gradient compute intensive tile of a column of compute intensive tiles between a first memory intensive tile and a second memory intensive tile.