Sparsity-Aware CiM MAC Hardware for High-Precision AI Acceleration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Compute cycle time in compute-in-memory (CiM) architectures is dominated by bit-serial MAC operations, which worsen for high precision data types like 16-bit floating point (FP16) and 32-bit floating point (FP32), leading to performance degradation and energy inefficiency due to unnecessary compute cycles for sparse workloads.
Innovation Solution
Implementing sparsity-aware digital CiM MAC hardware that restricts serial bit selection to non-zero values, using a leading 1's position detector and mask bits generator to skip zero bits during bit-serial compute, thereby reducing compute cycles and energy consumption without requiring pre-training or pruning of workload data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If bit-serial MAC operations are used in CiM architectures, then hardware complexity is reduced and memory integration is improved, but compute cycle time increases and performance deteriorates for high precision data types
Solution Approach 1:
The patent applies partial action by performing MAC operations only on non-zero input bits rather than all bits. The system detects leading 1's positions and skips processing of trailing zero bits, executing computations partially based on the actual sparsity pattern of the input data. This reduces the effective number of compute cycles needed while maintaining correctness of the MAC operation results.
Solution Approach 2:
The patent segments the bit-serial MAC operation into phases based on sparsity patterns. By identifying leading 1's positions and grouping consecutive non-zero bits, the system divides the computation into smaller segments that can be processed more efficiently, rather than treating all bits uniformly in a single sequential pass.
2Measurement precision
If bit-serial MAC operations process all input bits sequentially, then computational accuracy is maintained, but energy consumption increases due to unnecessary cycles for sparse workloads
Solution Approach 1:
The system performs partial MAC operations by skipping cycles for zero bits. The leading 1's position detector identifies where non-zero sequences begin, and the system processes only those necessary bits, reducing energy consumption proportionally to the sparsity of the input while maintaining full computational accuracy for the non-zero portions.
Solution Approach 2:
The patent changes the operational parameter of the MAC unit from fixed full-bit processing to variable bit-processing based on sparsity. By dynamically adjusting the number of compute cycles executed based on detected leading 1's positions and input sparsity patterns, the system optimizes energy consumption while preserving accuracy.
3Reliability
If full bit-serial processing is performed on all input data, then complete computational coverage is achieved, but compute cycles are wasted on sparse workloads with many zero values
Solution Approach 1:
The system applies partial action by executing MAC operations only for necessary non-zero bits. The leading 1's position detector and mask bits generator work together to identify and process only the relevant portions of input data, reducing wasted compute cycles on zero values while ensuring complete coverage of all contributing bits to the final result.
Solution Approach 2:
The patent implements skipping by allowing the MAC unit to rush through zero bits without executing full compute cycles. By detecting leading 1's positions and using mask bits to identify sparsity patterns, the system skips unnecessary processing of zero values and directly proceeds to process only the meaningful non-zero portions of the input data.
Data Source
AI summary
Systems, apparatuses and methods may provide for technology that includes a compute-in-memory (CiM) enabled memory array to conduct digital bit-serial multiply and accumulate (MAC) operations on multi-bit input data and weight data stored in the CiM enabled memory array, an adder tree coupled to the CiM enabled memory array, an accumulator coupled to the adder tree, and an input bit selection stage coupled to the CiM enabled memory array, wherein the input bit selection stage restricts serial bit selection on the multi-bit input data to non-zero values during the digital MAC operations.