Configurable Neural Processing Accelerator for Sparse Tensor Calculations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processing systems for neural network training and inference are costly and lack flexibility in performing various tensor calculations, particularly efficient sparse operations.
Innovation Solution
A system comprising a scratch memory connected to configurable processing elements via a communication fabric, allowing for flexible configuration and operation through configuration registers, node link circuits, and row caches, enabling efficient element-wise multiplications, additions, and accumulations, as well as conversion between dense and sparse representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional processing systems are used for neural network calculations, then general-purpose computing is maintained, but cost increases and flexibility for tensor calculations decreases
Solution Approach 1:
The processing system is divided into multiple independent processing elements (PEs) that can be individually configured and operated. Each PE is a self-contained unit with its own configuration register, allowing the system to be segmented into functional blocks that can be independently optimized for different tensor calculation tasks, reducing overall system cost while maintaining flexibility.
Solution Approach 2:
The processing elements are designed as universal units capable of performing multiple tensor calculation operations including element-wise multiplications, additions, and accumulations. By making each PE multi-functional through configurable operations, the system achieves high adaptability for various neural network calculations without requiring specialized hardware for each operation, thereby controlling device complexity.
2Adaptability or versatility
If fixed-architecture processing systems are used, then hardware simplicity is maintained, but adaptability to various neural network operations decreases
Solution Approach 1:
The processing elements incorporate configuration registers that allow dynamic reconfiguration of operation modes. Each PE can be programmed at runtime to perform different operations (element-wise multiplication, addition, accumulation) based on the specific neural network task requirements. This dynamic configurability provides high adaptability while maintaining relatively simple hardware architecture through a unified PE design.
Solution Approach 2:
The system achieves adaptability by changing operational parameters through configuration values stored in registers. By modifying the configuration parameters of processing elements rather than changing the physical hardware architecture, the system can adapt to various neural network operations. This parameter-based control maintains hardware simplicity while providing operational flexibility.
3Productivity
If parallel processing is implemented, then calculation speed increases, but system complexity and cost increase
Solution Approach 1:
The parallel processing capability is achieved by segmenting the system into multiple identical processing elements that operate simultaneously. Each PE is a simple, replicated unit that processes data in parallel, increasing calculation speed without requiring complex interconnections between different functional blocks. The segmentation into uniform PEs simplifies the parallel architecture compared to heterogeneous parallel systems.
Solution Approach 2:
Multiple processing elements are merged into a unified parallel processing array that shares common resources such as the communication fabric and configuration mechanism. By merging identical PEs into a coordinated parallel system with shared infrastructure, the architecture achieves high calculation speed while avoiding the complexity of fully distributed systems, as the PEs can be configured and controlled collectively.
Data Source
AI summary
A system for calculating. A scratch memory is connected to a plurality of configurable processing elements by a communication fabric including a plurality of configurable nodes. The scratch memory sends out a plurality of streams of data words. Each data word is either a configuration word used to set the configuration of a node or of a processing element, or a data word carrying an operand or a result of a calculation. Each processing element performs operations according to its current configuration and returns the results to the communication fabric, which conveys them back to the scratch memory.