Sparsity-Aware Compute-in-Memory Arrays for Lower-Power AI Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing complex machine learning tasks on lower power devices, such as edge devices, is challenging due to constraints on power consumption, computational efficiency, and memory footprint, even with efficient machine learning model architectures.
Innovation Solution
The method involves disabling bit cells in a compute-in-memory (CIM) array based on the sparsity of input data and weight data, resequencing data accordingly, processing the data with enabled bit cells, applying compensation for sparsity, and outputting the compensated values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If complex machine learning tasks are implemented on lower power devices using conventional methods, then computational accuracy is maintained, but power consumption and processing time are excessive
Solution Approach 1:
The patent extracts and removes redundant computational operations from the machine learning processing pipeline. By identifying and eliminating unnecessary computations in neural network layers, the system reduces power consumption and processing time while maintaining computational accuracy for the essential operations.
Solution Approach 2:
The patent applies partial action by selectively processing only the necessary portions of data through the CIM array. By disabling irrelevant bit cells and skipping unnecessary computational steps, the system performs just enough computation to maintain accuracy while significantly reducing energy consumption and processing time.
2Productivity
If efficient machine learning model architectures are used, then computational efficiency is improved, but device complexity increases
Solution Approach 1:
The patent implements self-service through automated sparsity detection and adaptive bit cell disabling mechanisms. The system automatically identifies sparse regions in the data and configures the CIM array accordingly without requiring complex external control logic, thereby improving computational efficiency while minimizing the increase in device complexity.
3Loss of energy
If sparsity-aware techniques are applied to reduce processing operations, then power consumption and processing time are reduced, but computational accuracy may be compromised
Solution Approach 1:
The patent employs feedback mechanisms where the sparsity detection unit continuously monitors the input data characteristics and dynamically adjusts the bit cell configuration accordingly. This closed-loop approach ensures that computational accuracy is maintained by adapting the processing strategy based on the actual sparsity patterns observed in the data, while still achieving energy savings.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for performing machine learning computations in a compute in memory (CIM) array comprising a plurality of bit cells, including: determining that a sparsity of input data to a machine learning model exceeds an input data sparsity threshold; disabling one or more bit cells in the CIM array based on the sparsity of the input data prior to processing the input data; processing the input data with bit cells not disabled in the CIM array to generate an output value; applying a compensation to the output value based on the sparsity to generate a compensated output value; and outputting the compensated output value.