Weight Expansion for Resilient MAC Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analog in-memory AI applications face accuracy issues due to noise sensitivity of weights and inputs in parallel computing operations, particularly in crossbar arrays constructed of non-volatile memory elements, where reducing precision impacts the accuracy of multiply-and-accumulate (MAC) operations.
Innovation Solution
A computer-implemented method generates resilient weights and inputs by providing a matrix and its pseudoinverse, multiplying weights with the pseudoinverse and input vectors to create new matrices with increased size and reduced precision, which are more resilient to noise and computational load, using techniques like Moore Penrose inversion and quantization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If precision of weights and inputs is reduced in MAC operations, then computational load is reduced and noise sensitivity is decreased, but accuracy of the computing operations deteriorates
Solution Approach 1:
The weight matrix W is segmented into two components: W = W1 + W2, where W1 contains the original weights and W2 contains the expanded weights with reduced precision. This segmentation allows the system to maintain high-precision core weights while using lower-precision expanded weights to compensate for noise, thereby resolving the contradiction between accuracy and noise sensitivity.
Solution Approach 2:
The patent creates a composite weight structure combining high-precision original weights and low-precision expanded weights. This composite approach allows the system to leverage the stability of high-precision weights while using the noise-resilient properties of low-precision weights, effectively balancing accuracy and noise sensitivity in the MAC operations.
2Reliability
If the number of weights is increased to compensate for reduced precision, then resilience to reduced precision is improved, but device complexity increases
Solution Approach 1:
The patent expands the weight representation by introducing a new dimension through the weight expansion matrix. By transforming the weight space from a single precision level to multiple precision levels (original weights + expanded weights), the system achieves resilience to precision reduction without proportionally increasing overall complexity, as the expanded weights can be stored and processed in a structured manner.
3Reliability
If weights are expanded with increased size to reduce precision sensitivity, then resilience to noise is improved, but memory requirements and computational overhead increase
Solution Approach 1:
The patent changes the precision parameter of weights by creating expanded weights with deliberately reduced precision. This parameter change allows the expanded weights to be less sensitive to noise and precision variations, achieving noise resilience while managing memory requirements through controlled precision reduction rather than simply increasing weight quantity.
Data Source
AI summary
A computer-implemented method to generate weights and inputs in a multiply-and-accumulate (MAC) operation that are resilient to reduced precision. The method includes providing a matrix M and its pseudoinverse M−1. Weights W are multiplied with M−1 and an input vector value x is multiplied with M. Two new matrices W2 and x2 are defined based on the multiplying of W with M−1 and x with M. The matrices W2 and x2 are encoded with increased resilience to reduce precision in place of the weights W and input vector value x.