CMOS Resistive Processing Unit for Asymmetric Weight Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing CMOS-based resistive processing units (RPUs) for neural network training require complex control circuits for symmetric weight updates, limiting speed and efficiency.
Innovation Solution
A CMOS-based RPU design that facilitates asymmetric weight updates using a capacitor connected to a transmission gate circuit and a readout field-effect transistor, simplifying the control circuit and enabling parallel operations through stochastic pulse generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If symmetric weight update control circuits are used in CMOS-based RPU, then weight update functionality is achieved, but circuit complexity increases and speed is limited
Solution Approach 1:
The patent applies asymmetry by designing separate control circuits for positive and negative weight updates instead of using symmetric control. The positive weight update uses a first control circuit with specific transmission gates, while the negative weight update uses a second control circuit with different configuration. This asymmetric design simplifies the overall control logic compared to symmetric approaches while enabling faster parallel operations during neural network training.
Solution Approach 2:
The control circuit is segmented into distinct positive weight update control circuit and negative weight update control circuit. Each segment handles specific update directions independently, allowing parallel operation without the complexity of unified symmetric control. This segmentation enables simultaneous positive and negative updates across different synaptic elements, improving overall training speed.
2Productivity
If complex control circuits are used for symmetric weight updates, then weight update accuracy is maintained, but training efficiency decreases
Solution Approach 1:
The patent implements asymmetric control where positive and negative weight updates are managed by different simplified control circuits rather than a single complex symmetric controller. This reduces overall circuit complexity while maintaining training efficiency through parallel execution of update operations.
Solution Approach 2:
The control circuit dynamically switches between positive and negative update modes using control signals that activate appropriate transmission gates. This dynamic control enables efficient parallel operations during different phases of neural network training without requiring complex static control logic.
3Speed
If parallel operations are enabled for weight updates, then training speed increases, but control circuit complexity increases
Solution Approach 1:
The parallel control capability is segmented into independent positive and negative update control paths. Each path has its own simplified control circuit that can operate independently and simultaneously, enabling parallel training operations without requiring a single complex control unit that would manage all parallel operations.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The design supports high signal-to-noise ratio and a high number of states, accelerating neural network training by simplifying control circuits and enabling parallel weight updates.
Implementation Method 1
a capacitor coupled to the transmission gate circuit and coupled to the readout field-effect transistor
Implementation Method 2
a readout field-effect transistor
Data Source
AI summary
A resistive processing unit to accelerate neural network training through asymmetric weight update. The resistive processing unit includes a readout field-effect transistor and a transmission gate circuit, the transmission gate circuit being coupled to a selectable first voltage source and including a first transmission gate and a second transmission gate, and having at least one transistor gate coupled to a second voltage source. A capacitor is coupled to the transmission gate circuit and is coupled to the readout field-effect transistor. An input of an inverter is coupled to at least one gate of the first transmission gate and an output of the inverter is coupled to at least one gate of the first transmission gate.