Two-Phase ANN Training Using Error-Prone MRAM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern artificial neural networks (ANNs) require significant energy and on-chip memory resources for training due to the large amount of data transfer and the need for error-free memory, which limits their deployment efficiency.
Innovation Solution
The use of error-prone magnetic random access memory (MRAM) for part of the training process, allowing for energy-efficient and area-reduced training by splitting the training iterations into two phases: an initial phase using error-prone MRAM for high-entropy exploration and a final phase using error-free memory for precision convergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If error-free memory is used for ANN training, then training reliability is improved, but energy consumption increases and on-chip area footprint increases
Solution Approach 1:
The training process is divided into two distinct phases: an initial high-entropy phase using error-prone MRAM for exploration, and a final convergence phase using error-free SRAM for precision. This segmentation allows each memory type to be optimized for its specific function, reducing overall energy consumption while maintaining training reliability.
Solution Approach 2:
Different memory types are assigned to different training phases based on their specific characteristics. MRAM's non-volatile and energy-efficient properties are leveraged for the initial exploration phase, while SRAM's high-speed and error-free properties are used for the final convergence phase. This local optimization of memory quality matches the specific requirements of each training stage.
2Reliability
If error-free memory is used for ANN training, then training reliability is improved, but on-chip area footprint increases
Solution Approach 1:
The training process is divided into two distinct phases: an initial high-entropy phase using error-prone MRAM for exploration, and a final convergence phase using error-free SRAM for precision. This segmentation allows each memory type to be optimized for its specific function, reducing overall energy consumption while maintaining training reliability.
Solution Approach 2:
Different memory types are assigned to different training phases based on their specific characteristics. MRAM's non-volatile and energy-efficient properties are leveraged for the initial exploration phase, while SRAM's high-speed and error-free properties are used for the final convergence phase. This local optimization of memory quality matches the specific requirements of each training stage.
3Measurement precision
If large amounts of data are transferred to and from chip, then training accuracy is improved, but energy consumption increases
Solution Approach 1:
The system performs preliminary exploration and parameter space search during the initial phase using MRAM, storing intermediate results non-volatily. This preliminary action reduces the need for repeated data transfers during later phases, as the convergence phase can work with previously computed intermediate results stored in MRAM, thereby reducing energy-consuming data transfers while maintaining training accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces energy consumption and on-chip memory requirements while maintaining training efficacy, enabling more efficient training of ANNs and potentially other optimization processes.
Implementation Method 1
a first random access memory (RAM) comprising magnetic random access memory (MRAM)
Data Source
AI summary
A computing device receives first data on which to train an artificial neural network (ANN). Using magnetic random access memory (MRAM), the computing device trains the ANN by performing a first set of training iterations on the first data. Each of the first set of iterations includes writing values for a set of weights of the ANN to the MRAM using first write parameters corresponding to a first write error rate. After performing the first set of iterations, the computing device performs a second set of training iterations on the first data. Each of the second set of iterations includes writing values for the set of weights of the ANN to the MRAM using second write parameters corresponding to a second write error rate. The second write error rate is lower than the first write error rate. The computing device stores values for the trained ANN.