Machine Learning Accelerator with Conditional MAC Skip Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning hardware accelerators face challenges in reducing latency and power consumption due to the need for extensive computations in determining activation values for neural network nodes, which can lead to increased silicon area and energy usage.
Innovation Solution
The system controls and decreases the number of computations by using input and weight properties to determine a retrieval condition, deciding which multiply and accumulate operations to perform or skip, based on the properties of inputs, weights, and the current value in the accumulation register, thereby optimizing MAC operations and memory access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all MAC operations are performed for every input-weight pair, then computation accuracy is maintained, but power consumption and latency increase
Solution Approach 1:
The patent applies partial action by selectively performing MAC operations only for input-weight pairs that satisfy the retrieval condition. The system evaluates properties (such as sparsity patterns, magnitude thresholds, or activation conditions) and executes computations only when necessary, rather than performing all possible MAC operations. This reduces power consumption while maintaining sufficient computation accuracy for the neural network inference task.
Solution Approach 2:
The system dynamically changes operational parameters by adjusting the retrieval condition based on input properties, weight properties, and accumulation register state. These parameter changes enable the system to adapt the computation intensity and memory access frequency, optimizing the trade-off between accuracy and power consumption for different input patterns and network layers.
2Measurement precision
If all MAC operations are performed for every input-weight pair, then computation accuracy is maintained, but latency increases
Solution Approach 1:
By performing only the necessary subset of MAC operations determined by the retrieval condition, the system reduces the total computation time and memory access latency while preserving the accuracy required for correct neural network inference results.
Solution Approach 2:
The system performs preliminary evaluation of input and weight properties before executing MAC operations. This preliminary action allows the system to identify and skip unnecessary computations in advance, reducing latency by avoiding wasted time on operations that would not contribute meaningfully to the final result.
3Use of energy by moving object
If retrieval condition checking is performed for every input-weight pair, then power consumption is reduced by skipping unnecessary operations, but device complexity increases
Solution Approach 1:
The retrieval condition checking mechanism is applied locally at each compute node rather than globally across the entire system. Each node independently evaluates its own input-weight pairs against the retrieval condition, allowing for modular implementation that limits the complexity increase to individual nodes while maintaining system-wide power savings.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This specification relates to systems, apparatus and methods for accelerating machine learning algorithms, matrix multiplications and/or digital signal processing. According to a first aspect of this disclosure, there is described Apparatus comprising means for performing operations comprising: retrieving, from memory, one or more input properties for respective inputs of a plurality of inputs to a compute node; retrieving, from memory, one or more weight properties for respective weights of a plurality of weights associated with the compute node, wherein a respective weight corresponds to one or more of the plurality of inputs to the compute node; for a sequence of pairs of values, each pair comprising an input value and a weight value: determining, based on one or more input properties of the input value, one or more weight properties of the weight value and properties of a current value of an accumulation register associated with the compute node, if a retrieval condition is satisfied: in the event of a positive determination: retrieving the input value and weight value of the pair from their respective memory and/or register; multiplying the retrieved input value and retrieved weight value to obtain a product value; and adding the product value to the accumulation register; in the event of a negative determination: refraining from retrieving the input value and weight value of the pair from their respective memories and/or registers; and determining an activation value of the compute node based on the current value of the accumulation register using an activation function.