Programmable Multiply-Add Array Hardware for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current CPU/GPU platforms are inadequate for supporting the growing computation demands of neural network-based deep learning applications in terms of performance, power efficiency, and economic scalability, particularly due to the high area and power costs of large-scale parallel multiply-accumulation structures required for neural network algorithms.
Innovation Solution
A software programmable architecture that enables a connection between a multiplier array and an adder array, allowing for the reuse of adders to perform either multiply-accumulate or multiply-reduce operations, which is more area- and power-efficient, especially in neural network processing units with numerous data lanes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If large-scale parallel multiply-accumulation structures are used to accelerate neural network computations, then computational speed is improved, but area and power consumption increase
Solution Approach 1:
The adder array is designed to serve multiple functions: it can perform accumulation operations during multiply-accumulate modes and reduce operations during multiply-reduce modes. This multi-functionality allows the same hardware resources to support different computational patterns, reducing the overall area required compared to having dedicated separate structures for each operation type.
Solution Approach 2:
The architecture employs dynamic reconfiguration of the adder array connections through software control. The connection pattern between multipliers and adders can be programmably changed to support different data flow configurations, enabling the system to adapt to various neural network computation patterns without requiring fixed dedicated hardware for each pattern.
2Productivity
If large-scale parallel multiply-accumulation structures are used to accelerate neural network computations, then computational speed is improved, but power consumption increases
Solution Approach 1:
By making the adder array multi-functional for both accumulation and reduction operations, the system avoids the need for separate dedicated hardware structures, thereby reducing total power consumption while maintaining computational performance.
Solution Approach 2:
The architecture allows the adder array to be dynamically reassigned between different operational modes. When not actively used for a particular function, the adder resources can be logically discarded from the active computation path and recovered for use in alternative modes, optimizing power efficiency by ensuring resources are actively utilized only when needed.
3Ease of manufacture
If fixed hardware architecture is used for multiply-accumulate operations, then implementation is simple, but adaptability to different computational patterns is limited
Solution Approach 1:
The architecture transitions from a fixed hardware configuration to a dynamically reconfigurable system where the connection pattern between multipliers and adders can be programmably changed. This allows the same hardware structure to adapt to different computational patterns required by various neural network algorithms.
Solution Approach 2:
The adder array is designed with universal functionality to support multiple operational modes including multiply-accumulate, multiply-reduce, and other variation modes. This universality is achieved through software-controlled connection reconfiguration, allowing a single hardware structure to handle diverse computational patterns.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An integrated circuit including a data architecture including N adders and N multipliers configured to receive operands. The data architecture receives instructions for selecting a data flow between the N multipliers and the N adders of the data architecture. The selected data flow includes the options: (1) a first data flow using the N multipliers and the N adders to provide a multiply-accumulate mode and (2) a second data flow to provide a multiply-reduce mode.