Neural Network Memory Architecture for Parallel Weight Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require significant computational and memory resources, leading to inefficiencies and increased operation times due to the need for multiple layers and operations, necessitating a solution to reduce the time elapsed for neural network operations.
Innovation Solution
A memory device with dedicated memories corresponding to each link in a neural network, storing weights and candidate weights, and a processing element to perform multiplication operations, allowing for simultaneous operations across batches without waiting for weight updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple layers are included in neural networks to obtain more accurate results, then the accuracy of neural network results is improved, but the time elapsed for operations increases
Solution Approach 1:
The patent divides the neural network into multiple independent processing elements (PEs), each responsible for specific layers or operations. This segmentation allows parallel execution of different neural network layers across multiple PEs, maintaining accuracy while reducing overall operation time through concurrent processing.
Solution Approach 2:
The patent implements weight update mechanisms where candidate weights are prepared and stored in advance in the dedicated memory. During forward propagation, these pre-prepared weights are immediately available for multiplication operations, eliminating waiting time and enabling continuous parallel processing across batches.
2Measurement precision
If forward propagation and back propagation are performed for training neural networks, then the training accuracy is improved, but the processing device cannot be used efficiently and operation time increases
Solution Approach 1:
The patent segments the processing device into multiple independent processing elements, each capable of performing both forward propagation and back propagation operations simultaneously on different batches or layers. This segmentation allows the processing device to maintain high utilization efficiency while completing comprehensive training operations.
Solution Approach 2:
The patent implements a continuous training pipeline where multiple batches are processed concurrently through different processing elements. While one PE performs forward propagation on a current batch, another PE simultaneously performs back propagation on a previous batch, ensuring continuous useful action and eliminating idle processing time.
3Power
If processing devices are used to perform neural network operations, then the computational capability is improved, but the time elapsed for operations increases due to sequential processing requirements
Solution Approach 1:
The processing device is segmented into multiple independent processing elements that can operate in parallel. Each PE handles specific computational tasks for different batches or layers, enabling the system to leverage full computational capability while reducing overall operation time through concurrent execution.
Solution Approach 2:
The patent introduces parallelism as a new dimension for processing by deploying multiple processing elements that operate simultaneously. This transforms the processing architecture from sequential (one-dimensional) to parallel (multi-dimensional), allowing computational capability to be fully utilized without increasing operation time.
Data Source
AI summary
A memory device is configured to perform neural network learning, the memory device including a first dedicated memory corresponding to a first link included in a neural network and configured to store a first forward propagation weight and at least one first candidate weight for the first link, and a first processing element (PE) configured to perform a multiplication operation between an input and the first forward propagation weight stored in the first dedicated memory, for the first link, in which the first forward propagation weight stored in the first dedicated memory is configured to be updated with one of the at least one first candidate weight after the multiplication operation for the first link corresponding to the first dedicated memory.