Error Tolerant AI Accelerators via Weight Decomposition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
AI accelerators face errors due to exposure to radiative environments, which existing radiation-hardened memory solutions attempt to address but at the cost of increased area and power consumption.
Innovation Solution
A compute engine with a compute-in-memory (CIM) hardware module and weight decomposition write circuitry that stores weight decomposition data, allowing for the reconstruction of weights and provision of a replacement matrix to mitigate errors, using radiation-hardened memory to enhance fault tolerance while minimizing overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If radiation-hardened memory is used to protect AI accelerators from radiative errors, then fault tolerance is improved, but memory area and power consumption increase by a factor of two to three
Solution Approach 1:
The patent segments the weight matrix into multiple sub-matrices and stores them in different memory banks. This allows the system to use standard memory instead of radiation-hardened memory while maintaining fault tolerance through architectural redundancy. The segmentation enables error detection and correction at the sub-matrix level without requiring full memory hardening.
Solution Approach 2:
The patent introduces an intermediary error detection and correction layer between the radiative environment and the memory system. This intermediary consists of parity bits and checksum mechanisms that detect and correct errors without requiring the memory itself to be radiation-hardened, thus avoiding the area and power penalty.
2Reliability
If radiation-hardened memory is used to protect AI accelerators from radiative errors, then fault tolerance is improved, but power consumption increases by a factor of two to three
Solution Approach 1:
By segmenting weights into sub-matrices stored in standard memory banks, the system avoids the high power consumption of radiation-hardened memory while maintaining error protection through distributed storage and parity-based error detection at the sub-matrix level.
Solution Approach 2:
The patent uses inexpensive parity bits and checksum mechanisms as disposable error-detection layers rather than investing in expensive radiation-hardened memory. These lightweight error-protection mechanisms consume minimal power compared to fully hardened memory systems.
3Reliability
If traditional radiation-hardened configurations are used, then fault tolerance is improved, but memory overhead increases significantly
Solution Approach 1:
The weight matrix is divided into multiple sub-matrices, each stored in separate memory banks with associated parity information. This segmentation reduces memory overhead compared to traditional hardening by applying error protection only to sub-matrices rather than requiring full duplication of entire weight sets.
Solution Approach 2:
The patent changes the error-protection parameter from full memory hardening to sub-matrix level parity protection. This parameter change reduces the overhead ratio from 2-3x to a much lower value by applying error detection only to critical sub-matrix structures rather than all memory contents.
Data Source
AI summary
A compute engine including a compute-in-memory (CIM) hardware module and weight decomposition write circuitry is described. The CIM hardware module stores weights corresponding to a matrix. The CIM hardware module is configured to perform a vector-matrix multiplication (VMM) for the matrix. The weight decomposition write circuitry is coupled with the CIM hardware module and is configured to store weight decomposition data corresponding to the matrix. The weight decomposition write circuitry is also configured to determine a replacement matrix for the matrix from the weight decomposition data and to provide the replacement matrix to the CIM hardware module.