Transformer Neural Networks in Resistive Memory for Parallel Compute
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional implementations of transformer neural networks are prone to slow inference due to serial processing of weight values, limiting their speed and parallelism in both training and inference operations.
Innovation Solution
Implementing a transformer neural network directly in memory using resistive memory arrays, where weight values are stored and accessed locally, allowing for concurrent computations across multiple memory cells, thereby increasing parallelism and reducing the need for serial data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If transformer neural network is implemented using traditional host-based processing, then the system can maintain simplicity in architecture, but the inference speed becomes slow due to serial processing of weight values
Solution Approach 1:
The patent merges the neural network computation function with the memory device by implementing the transformer neural network directly within the memory architecture. The memory device now serves dual purposes: storing weight values and performing neural network computations, eliminating the need for separate host processing and thereby improving inference speed while managing complexity through functional integration
Solution Approach 2:
The memory device is designed to perform multiple functions: it stores weight values for the transformer neural network and simultaneously executes computation operations. This multi-functionality allows the same hardware resource to handle both data storage and processing tasks, improving overall system efficiency and inference speed without requiring entirely separate processing units
2Productivity
If transformer neural network uses serial processing of weight values, then the implementation can remain simple, but the parallelism in training and inference operations is limited
Solution Approach 1:
The memory device is divided into multiple memory cells that can operate independently and concurrently. Each memory cell can store and process weight values in parallel, enabling simultaneous execution of multiple computation operations. This segmentation of the memory architecture into independent functional units directly increases parallelism in both training and inference operations
Solution Approach 2:
The patent transitions from one-dimensional serial processing to multi-dimensional parallel processing by utilizing the two-dimensional array structure of memory cells. Computations are performed across rows and columns of memory cells simultaneously, adding spatial dimensionality to the processing architecture and thereby enabling massive parallelism without proportionally increasing control complexity
3Loss of time
If weight values are transferred from host to memory device, then the memory device can access data locally, but the data transfer time and host burden increase
Solution Approach 1:
The weight values are pre-loaded into the memory device during initialization or training phases, so that during inference operations, the memory device can immediately access and process these values without requiring real-time transfers from the host. This preliminary action of data preparation eliminates ongoing data transfer delays and allows the memory device to operate independently and efficiently
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the speed and efficiency of transformer neural network operations by performing computations locally within the memory device, improving both training and inference parallelism and reducing the burden on the host processor.
Implementation Method 1
The resistive memory array can comprise resistive memory cells. The resistive memory cells can be utilized to perform computations used to implement a transformer neural network.
Data Source
AI summary
Apparatuses and methods can be related to implementing a transformer neural network in a memory. A transformer neural network can be implemented utilizing a resistive memory array. The memory array can comprise programmable memory cells that can be programed and used to store weights of the transformer neural network and perform operations consistent with the transformer neural network.