Neural Network Operator Fusion for Higher Accelerator Density
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for implementing neural network mapping functions on hardware accelerators result in low computing density, limiting the ability to process larger and more complex neural networks.
Innovation Solution
Fusing multiple operators in the data path of neural network mapping functions without altering functionality, reducing hardware area usage and increasing computing density by combining vector multiplication blocks into single operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If multiple operators are implemented separately in the data path of neural network mapping functions, then the hardware implementation is simpler and more modular, but the computing density is low and hardware area usage is high
Solution Approach 1:
The patent combines multiple separate operator implementations into a single fused operator that performs multiple functions (e.g., matrix multiplication, activation function, and normalization) in one hardware unit. This merging increases computing density by reducing the number of separate hardware blocks while maintaining the same computational functionality, directly resolving the contradiction between hardware simplicity and computing density
2Ease of manufacture
If multiple vector multiplication blocks are used in the data path, then each operation can be implemented independently, but the hardware area occupied is large
Solution Approach 1:
The patent creates a universal vector multiplication block that can perform multiple different operations by applying different transformation matrices. Instead of having separate dedicated blocks for each operation (matrix multiplication, convolution, etc.), a single reconfigurable block handles all operations, significantly reducing hardware area while maintaining independent implementability of different operations through software configuration
3Adaptability or versatility
If conventional operator implementation methods are used, then the hardware accelerator can process current neural networks, but it cannot efficiently handle larger and more complex neural networks
Solution Approach 1:
The patent implements dynamic operator fusion where the degree and type of operator combination are adjusted based on the specific neural network being processed. The system can dynamically reconfigure the hardware to fuse operators for small networks or keep them separate for larger networks, providing adaptability across different network sizes and complexities while maximizing computing density for each case
Data Source
AI summary
Embodiments of the present disclosure include systems and methods for fusing operators for neural network hardware accelerators. A plurality of vector multiplication operations in a data path of a mapping function included in a neural network are identified. The plurality of vector multiplication operations are combined into a single vector multiplication operation in the data path of the mapping function. A programmable integrated circuit (IC) is programmed to implement the mapping function of the neural network