Sparsity-Aware Compute-in-Memory for Low-Power ML Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing complex machine learning tasks on lower power devices, such as edge devices, is challenging due to constraints on power consumption, computational efficiency, and memory footprint, especially when dealing with large models and time-consuming computations.
Innovation Solution
The method involves disabling bit cells in a compute-in-memory (CIM) array based on the sparsity of input data and weight data, resequencing data accordingly, processing the data with enabled bit cells, applying compensation for sparsity, and outputting the compensated values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional CIM arrays process all input data without sparsity awareness, then complete computational accuracy is maintained, but processing power and time are excessively consumed
Solution Approach 1:
The system performs preliminary analysis of input data sparsity before processing, identifying zero-valued inputs that would waste computational resources. This preliminary action enables selective disabling of unnecessary bit cells and columns, achieving up to 50% processing time reduction while maintaining accuracy through compensation mechanisms.
Solution Approach 2:
The patent applies different processing treatments to different regions of the CIM array based on local sparsity characteristics. Specifically, columns corresponding to zero-valued inputs are disabled, while non-zero columns continue processing. This local differentiation optimizes resource utilization without compromising overall computational accuracy.
2Productivity
If sparsity-aware processing disables bit cells to reduce processing time, then productivity improves, but output accuracy may deteriorate due to skipped computations
Solution Approach 1:
The system implements feedback through compensation values that are calculated based on the number of disabled columns and applied to the final output. This feedback mechanism ensures that the output accurately reflects what the complete computation would have produced, maintaining reliability while achieving productivity gains through selective disabling of redundant operations.
Solution Approach 2:
The patent changes the parameter space by introducing compensation values that adjust the output based on sparsity characteristics. By modifying the output parameter through compensation rather than changing the core computation, the system maintains accuracy while improving efficiency.
3Reliability
If all bit cells remain enabled during processing, then computational completeness is ensured, but power consumption increases on low power devices
Solution Approach 1:
The system extracts and removes redundant computational operations by disabling bit cells and columns that correspond to zero-valued inputs. This extraction of unnecessary computations directly reduces power consumption while maintaining computational completeness through the compensation mechanism that accounts for the disabled operations.
4Reliability
If sparsity compensation is applied to maintain accuracy, then output reliability improves, but processing complexity increases
Solution Approach 1:
The compensation mechanism applies a simplified correction based on the count of disabled columns rather than performing complete computations for all inputs. This partial action approach achieves the necessary accuracy improvement without implementing excessive complexity, using a straightforward compensation formula instead of complex recalculation procedures.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for performing machine learning computations in a compute in memory (CIM) array comprising a plurality of bit cells, including: determining that a sparsity of input data to a machine learning model exceeds an input data sparsity threshold; disabling one or more bit cells in the CIM array based on the sparsity of the input data prior to processing the input data; processing the input data with bit cells not disabled in the CIM array to generate an output value; applying a compensation to the output value based on the sparsity to generate a compensated output value; and outputting the compensated output value.