Vertical-Domain SOT Cells for In-Memory AI Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current hardware implementations of deep neural networks (DNNs) for inference are limited in terms of energy efficiency, memory consumption, and computational speed due to reliance on traditional Von Neumann architecture, which requires significant data movement between memory and processors.
Innovation Solution
A deep neural network device utilizing spin-orbit torque (SOT) cells with a ferromagnetic layer comprising multiple magnetic domains and etch control layers to store weights, enabling matrix-vector multiplication within non-volatile memory cells, reducing the need for data movement and enhancing computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If traditional Von Neumann architecture is used for DNN inference, then device complexity is reduced and ease of manufacture is improved, but energy consumption increases and computational speed decreases
Solution Approach 1:
The patent merges memory and computation functions into a single compute-in-memory device. The crossbar array integrates weight storage (in memory cells) and matrix-vector multiplication (via conductance-based computation), eliminating the need for separate memory and processor components. This merging directly reduces energy consumption by eliminating data movement while accepting increased device complexity.
Solution Approach 2:
The crossbar array serves multiple functions: it stores weights in memory cells, performs matrix-vector multiplication through conductance calculations, and outputs results via read operations. This multi-functionality allows a single device to replace both memory and computation hardware, reducing overall system energy consumption despite increased complexity of the individual device.
2Loss of energy
If traditional Von Neumann architecture is used for DNN inference, then device structure is simplified, but data movement between memory and processor increases energy consumption
Solution Approach 1:
By merging memory and computation into the crossbar array, the patent eliminates the physical separation between memory and processor that causes data movement. Weights remain stored in the memory cells while computation occurs in-place through conductance-based operations, completely eliminating energy loss from data movement between components.
Solution Approach 2:
The memory cells perform computation using their own conductance properties without requiring external data movement. The conductance of each memory cell directly represents a weight value, and the natural flow of current through the crossbar array performs the multiplication operation, allowing the memory structure to serve its own computation needs without additional energy expenditure.
3Quantity of substance
If compute-in-memory hardware is used for DNN inference, then energy consumption is reduced and density is increased, but manufacturing precision requirements increase
Solution Approach 1:
The patent uses conductance as the physical parameter for weight representation, allowing continuous or multi-level weight values to be encoded in memory cell conductance. This parameter change from binary to analog/multi-state representation increases the information density per cell while the conductance-based computation naturally handles the precision requirements through the physical properties of the materials.
Solution Approach 2:
The memory cells utilize phase transitions or resistance state changes to encode different weight values. By transitioning between different conductance states (representing different weight magnitudes), the system achieves high density storage while the physical nature of these transitions provides inherent stability and readability, managing manufacturing precision through material physics rather than purely geometric control.
4Productivity
If compute-in-memory hardware is used for DNN inference, then computational speed is improved, but manufacturing complexity increases
Solution Approach 1:
The patent replaces traditional electronic computation (using transistors and logic gates) with a conductance-based computation system. The matrix-vector multiplication is performed through the natural physical behavior of current flow through resistive elements, eliminating the need for complex mechanical/electronic computation circuits and enabling faster operation while using established semiconductor manufacturing processes for the memory cells themselves.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The SOT cell-based DNN device achieves lower energy consumption, higher density, and faster inference operations by performing matrix-vector multiplication directly in memory, overcoming limitations of traditional architectures.
Implementation Method 1
each node of the n rows and m columns of nodes comprising a spin orbit torque (SOT) cell, the SOT cell comprising: a SOT layer
Implementation Method 2
a ferromagnetic (FM) layer comprising two or more domains
Implementation Method 3
the FM layer comprising two or more magnetic domains
Implementation Method 4
a plurality of etch control layers. The etch control layers have different etching rates and are used to create domain walls between the two or more magnetic domains
Data Source
AI summary
The present disclosure is generally related to a deep neural network (DNN) device comprising a plurality of spin-orbit torque (SOT) cells. The DNN device comprises an array comprising n rows and m columns of nodes, each row of nodes coupled to one of n first conductive lines, each column of nodes coupled to one of m second conductive lines, each node of the n rows and m columns of nodes comprising a plurality of SOT cells, each SOT cell comprising: a SOT layer, a ferromagnetic layer comprising two or more magnetic domains, and a plurality of etch control layers. The etch control layers have different etching rates and are used to create domain walls between the two or more magnetic domains. The DNN device further comprises a controller configured to store at least one corresponding of a neural network in each of the two or more magnetic domains.