Dual-Precision Analog Memory Cell for Neural Network Weight Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current analog nonvolatile memory elements suffer from limited precision, accuracy degradation due to dynamic range limitations, and are unsuitable for neural network applications due to asymmetric and nonlinear weight updates.
Innovation Solution
The implementation of dual-precision analog memory cells, comprising a high precision volatile memory element coupled with a low precision nonvolatile memory element, allowing for seamless transition between training and inference phases in neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If analog nonvolatile memory elements are used, then power consumption is reduced and storage duration is extended, but precision and accuracy degrade due to limited dynamic range
Solution Approach 1:
The memory system is segmented into two distinct components: a volatile memory element for high-precision weight storage during training, and a non-volatile memory element for low-power storage during inference. This segmentation allows each component to operate in its optimal regime, resolving the contradiction between precision and power consumption.
Solution Approach 2:
The system dynamically switches between volatile and non-volatile memory elements based on operational phase. During training, the volatile element provides high precision; during inference, the non-volatile element provides low power consumption. This dynamic switching resolves the contradiction by making the system adaptable to different operational requirements.
2Use of energy by moving object
If analog nonvolatile memory elements are used, then power consumption is reduced, but weight updates become asymmetric and nonlinear
Solution Approach 1:
The weight update function is segmented into two phases: training phase using volatile memory for precise, symmetric weight updates, and inference phase using non-volatile memory for power-efficient operation. This segmentation ensures that asymmetric and nonlinear weight update issues are confined to the training phase only.
Solution Approach 2:
The volatile memory element acts as a temporary, disposable working memory for training operations where precise weight updates are critical. After training, its contents are transferred to the non-volatile memory, and the volatile memory is reset. This approach accepts the high power consumption of volatile memory during training as a necessary investment for achieving accurate weight updates.
3Device complexity
If single-precision memory elements are used, then device complexity is reduced, but accuracy degrades for neural network applications
Solution Approach 1:
The memory system is segmented into dual-precision components that work together: the volatile memory element provides high precision for training accuracy, while the non-volatile memory element provides durability for inference. This segmentation achieves high accuracy without requiring a single overly complex high-precision non-volatile memory element.
Solution Approach 2:
The system uses a composite memory architecture combining volatile and non-volatile memory elements, each with complementary strengths. This composite approach achieves high accuracy and reliability that neither component could provide alone, resolving the contradiction between device complexity and accuracy.
Data Source
AI summary
Dual-precision analog memory cells and arrays are provided. In some embodiments, a memory cell, comprises a non-volatile memory element having an input terminal and at least one output terminal; and a volatile memory element having a plurality of input terminals and an output terminal, wherein the output terminal of the volatile memory element is coupled to the input terminal of the non-volatile memory element, and wherein the volatile memory element comprises: a first transistor coupled between a first supply and a common node, and a second transistor coupled between a second supply and the common node; wherein the common node is coupled to the output terminal of the volatile memory element; and wherein gates of the first and second transistors are coupled to respective ones of the plurality of input terminals of the volatile memory element.