Multi-Precision MAC Architecture for Low-Power DNN Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in computational demands, as each test input involves hundreds of millions of MAC operations and filter weights, making high-end GPUs unsuitable for energy-constrained mobile/embedded systems due to high power consumption.
Innovation Solution
A multi-precision MAC unit-based processor that supports INT8/4/2/1 precision modes with shared circuit elements, reducing area and energy overhead through sparsity acceleration and input channel-based accumulation, enabling efficient operation of DNNs in mobile/embedded systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If high-end GPUs are used to execute DNN workloads, then computational throughput is improved, but power consumption increases significantly making them unsuitable for mobile/embedded systems
Solution Approach 1:
The GPU architecture is segmented into multiple independent MAC units that can operate in parallel. Each MAC unit is a discrete functional block that performs multiply-accumulate operations independently, allowing the system to scale throughput by activating only the necessary number of units rather than powering up the entire GPU.
Solution Approach 2:
The MAC units are designed with multi-precision support (INT8, INT4, INT2, INT1) that allows the same hardware circuitry to perform different precision operations. This universality reduces the need for separate dedicated circuits for each precision level, thereby reducing overall power consumption while maintaining flexibility to handle various DNN workload requirements.
2Adaptability or versatility
If multiple precision modes (INT8/4/2/1) are supported with dedicated circuits, then computational versatility is improved, but area overhead increases
Solution Approach 1:
A single MAC unit circuit is designed to support multiple precision modes (INT8, INT4, INT2, INT1) through configurable operation. The same multiply-accumulate circuitry can be dynamically configured to operate at different precision levels, eliminating the need for separate dedicated circuits for each precision mode and significantly reducing the total area required.
Solution Approach 2:
The MAC unit incorporates dynamic configuration capabilities that allow it to switch between different precision modes during operation. Control logic dynamically adjusts the operational parameters of the MAC unit based on the required precision level, enabling one circuit to adaptively serve multiple precision requirements without requiring static dedicated hardware for each mode.
3Device complexity
If traditional MAC units are used without sparsity acceleration, then circuit simplicity is maintained, but energy efficiency decreases in mobile/embedded systems
Solution Approach 1:
Sparsity detection and acceleration logic is integrated into the MAC unit to identify zero-valued inputs before they undergo full multiply-accumulate operations. By detecting sparsity patterns in advance, the system can skip unnecessary computational steps for zero inputs, reducing energy consumption without significantly increasing circuit complexity.
Solution Approach 2:
The MAC unit incorporates self-optimizing features where the sparsity acceleration logic automatically identifies and exploits zero-patterns in the input data. The circuit monitors its own operational patterns and dynamically adjusts its behavior to skip redundant operations when sparsity is detected, enabling energy efficiency improvements without requiring external control or significantly complicating the base circuit design.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, apparatuses and methods may provide for multi-precision multiply-accumulate (MAC) technology that includes a plurality of arithmetic blocks, wherein the plurality of arithmetic blocks each contain multiple multipliers, and wherein the logic is to combine multipliers one or more of within each arithmetic block or across multiple arithmetic blocks. In one example, one or more intermediate multipliers are of a size that is less than precisions supported by arithmetic blocks containing the one or more intermediate multipliers.