Dynamic Energy Saving Controller for ML Hardware Accelerators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning hardware accelerators for neural networks are power-hungry and latency-critical, with many unnecessary computations and memory accesses due to sparsity in operands and feature maps.
Innovation Solution
A dynamic energy saving controller that determines zero status bits, combined status bits, and pointers to skip unnecessary computations and memory accesses by exploiting activation and feature sparsity, using zero-gating and zero-skipping methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If machine learning hardware accelerators perform all computations including zero operations, then computational completeness is maintained, but power consumption increases significantly
Solution Approach 1:
The patent extracts and removes zero operations from the computational pipeline by determining zero status bits of operands and using combined status bits to identify when multiplication results will be zero, thereby skipping unnecessary computations and reducing power consumption while maintaining computational correctness
Solution Approach 2:
The patent performs preliminary determination of zero status bits and combined status bits before executing multiplication operations, allowing the system to pre-identify and skip zero operations, thus reducing overall power consumption without affecting computational completeness
2Use of energy by moving object
If the system skips zero computations to save power, then power consumption decreases, but computational accuracy may be compromised
Solution Approach 1:
The patent uses feedback from zero status bit determinations and combined status bit calculations to dynamically control computation execution, ensuring that only appropriate operations are skipped while maintaining computational accuracy through verified status information
3Productivity
If zero status bits and combined status bits are determined for all element pairs, then computation skipping efficiency increases, but device complexity increases
Solution Approach 1:
The patent segments the complexity by processing zero status bits and combined status bits in parallel for multiple element pairs simultaneously, distributing the control logic work across parallel hardware units rather than sequential processing, thus improving productivity without proportionally increasing overall device complexity
4Loss of time
If memory accesses are skipped for zero operations, then power consumption and latency decrease, but data availability for subsequent operations may be affected
Solution Approach 1:
The patent performs preliminary determination of zero status bits before memory accesses, allowing the system to skip memory reads for zero operations and reduce latency while maintaining data availability for non-zero operations through proper control logic
Data Source
AI summary
An apparatus includes means for: determining a zero status bit of a first element that indicates whether the first element is zero; determining a zero status bit of a second element that indicates whether the second element is zero; determining a combined status bit corresponding to an index of the first element and an index of the second element, wherein the combined status bit indicates whether a product of a next first element and a next second element is non-zero; determining a pointer that points to a memory address of a next set of elements comprising the next first element and the next second element, based on the combined status bit; retrieving the next set of elements from a location in the at least one memory given by the memory address; and performing a computation to determine the product of the next first element and the next second element.