Systolic Adder Tree Modes for Low-Power Convolution Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing adder tree structures for embedded neural processing units (NPUs) achieve low power consumption but suffer from decreased throughput due to reduced utilization of multiplier accumulators during depthwise convolution, while systolic array structures consume excessive power and are inefficient for general convolution operations.
Innovation Solution
A systolic adder tree structure with input multiplexers that adjust data movement and operation modes based on frequency and voltage, combined with a non-linear function module, to optimize power efficiency and resource utilization for both depthwise and general convolution operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If an adder tree structure is used for low-power driving in embedded NPU, then power consumption is reduced, but throughput decreases due to reduced utilization of multiplier accumulators during depthwise convolution
Solution Approach 1:
The patent implements dynamic operation modes (depthwise mode, general mode, and elementwise mode) that allow the adder tree structure to adapt its functionality based on the type of convolution operation being performed. This dynamic reconfiguration enables the system to maintain low power consumption while achieving high throughput for different operation types by optimizing resource utilization accordingly
Solution Approach 2:
The adder tree structure is designed to perform multiple functions: it can operate as a traditional adder tree for general convolution, as a depthwise convolution unit with reduced MAC utilization, or as an elementwise operation unit. This multi-functionality allows a single structure to handle various operation types efficiently without requiring separate dedicated hardware for each operation type
2Area of stationary object
If an architecture is designed to support only depthwise operation with reduced area, then area is reduced, but power consumption increases for elementwise add/multiply operations due to lack of dedicated hardware
Solution Approach 1:
The architecture dynamically switches between operation modes based on the input operation type. When elementwise operations are detected, the system activates the elementwise mode that utilizes the full adder tree structure for both multiplication and addition, thereby reducing power consumption without requiring separate dedicated hardware for each operation type
3Productivity
If a systolic array structure is used for general convolution operations, then throughput is high, but power consumption increases
Solution Approach 1:
The patent applies local quality by implementing different operational characteristics in different parts of the adder tree structure. For general convolution operations, the full MAC units are utilized throughout the structure to achieve high throughput. For depthwise operations, only the addition functionality is actively used in certain stages, reducing power consumption. This localized optimization allows the system to achieve high throughput when needed while consuming less power when full throughput is not required
Data Source
AI summary
An apparatus with deep learning includes: a systolic adder tree including adder trees connected in row and column directions; and an input multiplexer connected to an input register of at least one of the adder trees and configured to determine column directional data movement between the adder trees based on operation modes.