PIM Memory Quantization and Dequantization to Reduce Neural Network Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require significant computational resources and memory bandwidth, making them costly for organizations that cannot afford the hardware requirements, limiting their ability to utilize these networks effectively.
Innovation Solution
Implementing a memory array with PIM accelerators and scatter/gather circuits to perform quantization and dequantization operations, reducing the need for external processors to handle these tasks, and storing data in a single precision to minimize replication and bandwidth demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high precision data storage and processing is used for neural networks, then accuracy is improved, but hardware cost and computational resources increase significantly
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting data precision requirements based on the specific neural network operation being performed. Different precision levels (e.g., 8-bit, 16-bit, 32-bit) are used for different data types and computational stages, allowing the system to maintain accuracy where needed while reducing hardware complexity where high precision is not critical.
Solution Approach 2:
The patent segments the neural network computational pipeline into distinct stages with different precision requirements. By dividing the processing into separate functional units that handle different precision levels independently, the system achieves high accuracy for critical computations while using lower precision for less critical operations, thereby reducing overall hardware complexity.
2Productivity
If high computational power is provided for neural network processing, then processing speed is improved, but power consumption increases
Solution Approach 1:
The patent implements dynamic precision adjustment that adapts computational resources based on the specific operation being performed. By dynamically changing precision levels during different computational stages, the system maintains high processing speed for critical operations while reducing power consumption during less critical processing stages.
Solution Approach 2:
The patent applies different precision levels to different parts of the neural network computation based on their specific requirements. Critical computational paths use higher precision for speed and accuracy, while less critical paths use lower precision to reduce power consumption, achieving local optimization of the energy-speed tradeoff.
3Adaptability or versatility
If data is stored in multiple precisions to support neural network operations, then computational flexibility is improved, but memory bandwidth and storage requirements increase
Solution Approach 1:
The patent implements a universal memory architecture that can efficiently handle multiple precision formats through a single unified interface. The memory system is designed to accommodate different precision requirements (8-bit, 16-bit, 32-bit) without requiring separate storage paths, thereby maintaining computational flexibility while optimizing memory bandwidth utilization through consolidated access mechanisms.
Data Source
AI summary
An apparatus and method for creating less computationally intensive nodes for a neural network. An integrated circuit includes a host processor and multiple memory channels, each with multiple memory array banks. Each of the memory array banks includes components of a processing-in-memory (PIM) accelerator and a scatter and gather circuit used to dynamically perform quantization operations and dequantization operations that offload these operations from the host processor. The host processor executes a data model that represents a neural network. The memory array banks store a single copy of a particular data value in a single precision. Therefore, the memory array banks avoid storing replications of the same data value with different precisions to be used by a neural network node. The memory array banks dynamically perform quantization operations and dequantization operations on one or more of the weight values, input data values, and activation output values of the neural network.