In-Memory Pooling Circuitry for CNN Power Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNN) operations in machine learning, which rely on data movement between GPUs and DRAM, consume significant power and slow down performance due to the need for frequent data transfer through conductive traces and pads on printed circuit boards.
Innovation Solution
A device with a first block of memory cells storing an input array and a second block of memory cells, where pooling circuitry executes in-place pooling to generate output values as analog levels, eliminating the need for verify cycles and reducing power consumption by storing output values as analog levels instead of digital values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is moved between GPUs and DRAM for convolution and pooling operations, then the operations can be performed, but power consumption increases and performance slows down
Solution Approach 1:
The patent combines memory storage and processing functions into a single integrated memory device. The first block of memory cells stores input arrays while the second block stores output arrays, and pooling circuitry is integrated within the memory device to perform pooling operations directly on the stored data, eliminating the need for separate data movement between external GPUs and DRAM.
Solution Approach 2:
The patent introduces an integrated memory device as an intermediary between traditional DRAM and processing units. This memory device with embedded pooling circuitry acts as a mediator that performs both data storage and pooling operations, reducing the need for frequent data transfers to external components and thereby lowering power consumption while maintaining performance.
2Productivity
If data is frequently moved between multiple GPUs and DRAM through conductive traces and pads, then convolution and pooling operations can be executed, but the data movement consumes significant power and slows down performance
Solution Approach 1:
The patent segments the memory device into multiple functional blocks: a first block of memory cells for storing input arrays, a second block of memory cells for storing output arrays, and integrated pooling circuitry. This segmentation allows different operations to occur in parallel within the same device, reducing data movement time and improving overall performance speed.
3Reliability
If output values are stored as digital values, then verification can be performed, but verify cycles are needed which reduces performance
Solution Approach 1:
The patent changes the parameter of data representation from digital values to analog levels. By storing output values as analog levels in the second block of memory cells, the system eliminates the need for verify cycles while maintaining sufficient reliability for machine learning operations, thereby improving storing performance without sacrificing essential verification capabilities.
Data Source
AI summary
A method comprises a first block of memory cells to store an input array, and a second block of memory cells. Pooling circuitry is operatively coupled to the first block of memory cells to execute in-place pooling according to a function over the input array to generate an array of output values. Writing circuitry is operatively coupled to the second block to store the array of output values in the second block of memory cells. Analog sensing circuitry is coupled to the first block of memory cells to generate analog values for the input array, wherein the pooling circuitry receives the analog values as inputs to the function. The writing circuitry operatively coupled to the second block is configured to store an analog level in each cell of the second block for the array of output values.