Charge-Storing Synapse Circuit for Linear On-Chip DNN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing synapse devices for deep neural network training lack linear and symmetric weight updates, sufficient retention time, and parallel on-chip training operations, and suffer from device non-idealities such as drifting references and long-term retention loss, limiting software-level on-chip learning accuracy.
Innovation Solution
A synapse device with a 6T1C structure, comprising a weight capacitor and four control transistors, allows for linear and symmetric weight updates, providing sufficient retention time and parallel on-chip training operations, and compensates for device non-idealities through device-algorithm co-optimization using efficient training algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If resistance change memory, phase change memory, or ferroelectric memory is used to implement neural networks in hardware, then energy efficiency is improved, but computational accuracy lags behind software-based deep neural networks due to non-ideal characteristics of experimental memory devices
Solution Approach 1:
The patent changes the fundamental parameter from resistance-based storage to charge-based storage using capacitors. This allows the synapse device to achieve both high energy efficiency (comparable to resistive memory) and high computational accuracy (matching software-level precision) by utilizing voltage storage and controlled charge transfer mechanisms rather than relying on non-ideal resistance changes in experimental memory devices
Solution Approach 2:
The patent substitutes physical resistance change mechanisms with electrical charge storage and transfer mechanisms. By using capacitors to store voltage and controlled current flow to represent synaptic weights, the system replaces the non-ideal physical resistance changes with more controllable and precise electrical parameters, enabling both energy efficiency and computational accuracy
2Extent of automation
If conventional synapse devices are used for deep neural network training, then hardware implementation is achieved, but linear and symmetric weight updates are not provided, limiting software-level on-chip learning accuracy
Solution Approach 1:
The patent deliberately designs symmetric weight update characteristics by using matched capacitor pairs and balanced current mirror circuits. The differential architecture ensures that potentiation and depression operations are symmetric, and the linear relationship between input voltage and weight change is maintained through careful circuit design, enabling accurate gradient descent for on-chip learning
Solution Approach 2:
The patent changes the weight update mechanism from non-linear resistance changes to linear voltage-based charge transfer. By controlling the amount of charge transferred to capacitors in proportion to the input signal, the system achieves linear weight updates that accurately reflect the gradient descent calculations, enabling software-level learning accuracy on hardware
3Speed
If volatile memory devices like SRAM and DRAM are used, then high speed and wide bandwidth are achieved, but non-volatility and data preservation are lost
Solution Approach 1:
The patent merges the advantages of volatile and non-volatile memory by using capacitors that can rapidly charge and discharge (providing fast read/write speeds) while maintaining stored charge for extended periods when not accessed (providing non-volatility). The differential capacitor structure with controlled charge transfer enables both high-speed learning operations and stable weight retention during inference
Solution Approach 2:
The patent implements a continuous charge transfer mechanism where synaptic weights are maintained as stored charge on capacitors during inference operations, eliminating the need for repeated refresh operations. The charge is continuously preserved through the capacitor's inherent storage capability, providing both speed and retention simultaneously
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The 6T1C synapse device enables software-level on-chip learning with improved accuracy by implementing symmetrical and linear training characteristics, reducing accuracy deterioration due to asymmetry and leakage, and enhancing integration and robustness.
Implementation Method 1
a weight capacitor having a first terminal and a second terminal and storing a voltage corresponding to the weight
Implementation Method 2
four control transistors which change a voltage or a weight of the capacitor
Implementation Method 3
a first output transistor including a first gate terminal coupled to the first terminal of the capacitor, and outputting a first drain current according to a voltage applied to the first gate terminal
Data Source
Figure 1A
Figure 1B
Figure 2A~3A
AI summary
A charge-storing synapse device for deep neural network training and a driving method thereof are disclosed. The disclosed synapse device may include a weight capacitor having a first terminal and a second terminal and storing voltage corresponding to the weight; four control transistors which change a voltage or a weight of the capacitor; a first output transistor including a first gate terminal coupled to the first terminal of the capacitor, and outputting a first drain current according to a voltage applied to the first gate terminal; and a second output transistor including a second gate terminal coupled to the second terminal of the capacitor, and outputting a second drain current according to the voltage applied to the second gate terminal, and wherein the weight may be determined by the difference between the first drain current and the second drain current.