Systolic Array Neural Network Computation with Selective Element-Wise Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays are inefficient for element-wise operations in neural networks, leading to resource wastage when executing neural networks that do not require element-wise operations, such as MLP or CNN, as they typically include a dedicated block for element-wise operations which is not utilized.
Innovation Solution
A neural network computation apparatus with a systolic array that includes both matrix multiplication and element-wise operation processing units, allowing for selective performance of matrix multiplication and element-wise operations based on the neural network algorithm, eliminating the need for a separate dedicated element-wise operation block.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a dedicated element-wise operation block is added to the systolic array, then element-wise operations can be performed, but device complexity increases and resources are wasted when element-wise operations are not needed
Solution Approach 1:
The processing units in the systolic array are designed to perform multiple functions - both matrix multiplication operations and element-wise operations. Each processing unit can be configured to execute different operation types based on the neural network algorithm being run, eliminating the need for separate dedicated blocks for element-wise operations and thereby reducing device complexity while maintaining versatility
2Adaptability or versatility
If a dedicated element-wise operation block is always provided, then element-wise operations are supported, but resource utilization decreases when executing neural networks that do not require element-wise operations
Solution Approach 1:
The systolic array implements dynamic operation mode switching where processing units can adaptively change their operation type based on the computational requirements of the current neural network algorithm. This dynamic reconfiguration allows the system to activate element-wise operation capabilities only when needed, preventing resource waste during matrix multiplication-only workloads while maintaining support for both operation types
Data Source
AI summary
A neural network computation apparatus includes a first processing block including a plurality of processing units that each perform a matrix multiplication operation on input data and weights, and a second processing block including a plurality of element-wise operation processing groups. The element-wise operation processing group selectively perform a first neural network computation operation and a second neural network computation operation. The first neural network computation operation comprises the matrix multiplication operation on the input data and the weights and an activation operation on a result value of the matrix multiplication operation, and the second neural network computation operation comprises an activation operation on the result value of the matrix multiplication operation, which is transferred from the first processing block, and an element-wise operation.