MTJ Hardware Synapse for Binary and Ternary DNNs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks (DNNs) are compute-intensive and power-hungry, limiting their performance on low-power devices, and require frequent memory accesses, leading to high power consumption and execution latency.
Innovation Solution
The implementation of a magnetic tunnel junction (MTJ) based hardware synapse device that performs a gated XNOR (GXNOR) operation, allowing for ternary or binary synapse weights to be stored and updated in-situ, reducing memory access and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If digital MAC circuits are used for DNN computation, then computing capability is improved, but power consumption increases and execution latency increases due to frequent memory accesses
Solution Approach 1:
The patent merges memory and computation functions into a single MTJ-based synapse device. The MTJ devices store synapse weights in their resistance states while simultaneously participating in GXNOR operations to compute weighted sums, eliminating the need for separate memory reads and digital MAC operations. This in-situ computation approach resolves the contradiction by performing computing capability enhancement without the associated memory access overhead and power consumption.
Solution Approach 2:
The patent replaces the mechanical/digital MAC operation system with a magnetic-based computation system. Instead of using digital circuits to perform multiply-accumulate operations, the system uses MTJ devices where resistance states represent weights and current flows represent computations. This substitution eliminates frequent memory accesses and reduces power consumption while maintaining computing capability for DNN operations.
2Measurement precision
If large data structures are used for synapse and activations, then computational precision is improved, but memory capacity requirements increase and memory access frequency increases
Solution Approach 1:
The MTJ-based synapse devices serve dual purposes: they store synapse weights in their resistance states and simultaneously perform GXNOR computations. The devices self-configure their resistance states through stochastic switching during training, eliminating the need for external memory to store large weight matrices. This self-service approach maintains computational precision while dramatically reducing memory capacity requirements.
Solution Approach 2:
The patent changes the physical state parameters of MTJ devices to represent computational data. Resistance states (high/low) represent binary or ternary weight values, and current flow parameters represent activation values. By encoding computational precision in physical device states rather than digital memory, the system achieves high precision with minimal memory capacity.
3Use of energy by moving object
If discrete neural networks with binary or ternary weights are used, then power consumption is reduced, but implementation complexity increases due to stochastic update requirements
Solution Approach 1:
The patent replaces complex digital stochastic update circuits with intrinsic magnetic stochastic switching. Instead of using digital logic to generate stochastic updates, the system leverages the natural stochastic switching behavior of MTJ devices when subjected to current pulses. This substitution maintains the power efficiency of discrete neural networks while dramatically simplifying the implementation of stochastic update mechanisms.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables efficient execution of weight-related computations in deep neural networks, particularly in ternary and binary neural networks, on low-power devices with reduced power consumption and latency, achieving similar accuracy to ideal GXNOR algorithms with minor accuracy loss.
Implementation Method 1
first and second magnetic tunnel junction (MTJ) devices, wherein each of the MTJ devices has a fixed layer port and a free layer port
Implementation Method 2
the synapse device is configured to store a ternary or binary synapse weight represented by a state of the MTJ devices
Data Source
AI summary
A stochastic synapse for use in a neural network, comprising: first and second magnetic tunnel junction (MTJ) devices, each MTJ device having a fixed layer port and a free layer port; a first and second control circuit, each connected respectively to the free layer port of the first and second MTJ devices, wherein the fixed layer ports of the first and second MTJ devices are connected to each other; wherein the first and second control circuits are configured to perform a gated XNOR operation between synapse and activation values; and wherein an output of the gated XNOR is represented by the output current through both of the first and second MTJ devices.