FeFET In-Memory Computing for Synaptic Weight and Gradient Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-memory computing systems face inefficiencies in training machine learning models due to limitations in synaptic weight representation and gradient accumulation, particularly in neuromorphic computing systems using resistive processing units.
Innovation Solution
The implementation of non-volatile resistive memory elements, specifically ferroelectric field-effect-transistors (FeFETs), which store synaptic weights and gradient accumulation values, enabling efficient in-memory computing operations through forward and backward passes to optimize machine learning model training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional resistive processing units are used for in-memory computing, then basic computing operations can be performed, but training efficiency and reliability of machine learning models deteriorate due to limitations in synaptic weight representation and gradient accumulation
Solution Approach 1:
The patent divides the RPU into two separate FeFET devices: one dedicated to storing synaptic weight values and another dedicated to accumulating gradient values. This segmentation allows each FeFET to be optimized for its specific function, improving both the accuracy of synaptic weight representation and the efficiency of gradient accumulation during model training
Solution Approach 2:
The patent enables the RPU to perform multiple functions simultaneously by using two FeFETs that can independently store different conductance values. One FeFET handles synaptic weight storage while the other handles gradient accumulation, allowing the system to perform both forward pass computations and backward pass gradient updates without external intervention, thereby improving training efficiency
2Adaptability or versatility
If a single FeFET is used per RPU, then device complexity is reduced, but the ability to simultaneously store and update synaptic weights and gradients deteriorates
Solution Approach 1:
The patent segments the storage function into two distinct FeFET devices within each RPU. This segmentation enables simultaneous storage of synaptic weights and gradients, as well as independent update operations, without requiring complex external control circuitry. The increased device complexity of adding one more FeFET is offset by the elimination of complex external memory and control structures
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances the efficiency and reliability of in-memory computing by accurately updating synaptic weights and gradients, thereby improving the training process of machine learning models.
Implementation Method 1
a first ferroelectric layer... a second ferroelectric layer
Data Source
AI summary
A computer program product for training a machine learning model. A processor executes program instructions stored on a computer readable media to perform a forward pass read by applying a bias input voltage to a gate of a first resistive processing unit (RPU) configured for infrequent writes and storing a first conductance value representing a synaptic weight value to read from the first RPU a product of the bias input voltage and the first conductance value. The processor performs a backward pass to compute a loss value for the forward pass read and to compute a gradient value to minimize the loss value. The processor applies a coercive input voltage to a gate of a second RPU configured differently for frequent writes to write to the second RPU a second conductance value related to the gradient value and representing a gradient accumulation value.