Computational Memory Banks with Embedded SIMD Controllers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computational memory architectures for deep learning and neural networks face inefficiencies such as high power consumption, increased complexity, and larger chip area requirements due to the need for processing large numbers of computations, which are exacerbated by the distance between memory and processing elements, and are inadequate for handling two-dimensional images and flexible arithmetic operations.
Innovation Solution
The implementation of a computational memory system with a plurality of banks, each containing an array of memory units and processing elements connected by column and row buses, along with single instruction, multiple data (SIMD) controllers, to facilitate efficient processing and communication, allowing for flexible low-precision arithmetic and local storage of instructions and coefficients, thereby reducing power consumption and increasing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If data is physically moved between memory and processing elements, then computation can be performed, but power consumption increases due to charging and discharging wiring capacitance
Solution Approach 1:
The patent combines memory and processing elements into a unified computational memory structure where processing elements are directly embedded within the memory array. This merging eliminates the need for separate data transfer paths, thereby reducing the wiring capacitance that must be charged and discharged, and consequently lowering power consumption.
Solution Approach 2:
The patent introduces local processing elements as intermediaries within the memory structure itself. These processing elements perform computations directly at the memory location, acting as intermediaries that eliminate the need for data to be physically moved to separate processing units, thereby reducing the energy loss associated with charging and discharging long wiring paths.
2Quantity of substance
If processing elements are located far from memory, then memory capacity can be increased, but the distance between memory and processing elements increases power consumption
Solution Approach 1:
The patent transitions from a traditional von Neumann architecture where processing elements are separated from memory in a linear fashion to a two-dimensional or three-dimensional integrated structure. Processing elements are embedded within the memory array in additional spatial dimensions, allowing both large memory capacity and short distances to be achieved simultaneously.
Solution Approach 2:
The patent implements a nested structure where processing elements are embedded within the memory array, which itself is organized in a hierarchical structure. This nesting allows processing elements to be positioned immediately adjacent to the memory cells they operate on, eliminating the need for long-distance data transfer while maintaining large memory capacity through hierarchical organization.
3Productivity
If traditional memory architecture is used for neural networks, then general memory functions are provided, but processing efficiency decreases and chip area increases
Solution Approach 1:
The patent creates a multi-functional computational memory structure where the same physical components serve both as memory storage and as processing units. The processing elements embedded within the memory array can perform both data storage and neural network computations, eliminating the need for separate memory and processing areas on the chip.
Solution Approach 2:
The patent merges memory and processing functions into a single integrated structure, where processing elements are directly embedded within the memory array. This consolidation eliminates the need for separate memory and processing areas, thereby reducing chip area while simultaneously improving processing efficiency by eliminating data transfer overhead.
4Adaptability or versatility
If more computations are performed for deep learning, then neural network functionality is achieved, but power consumption increases
Solution Approach 1:
The patent implements a self-service architecture where processing elements embedded within the memory array perform computations directly on the stored data without requiring external data movement. This self-service capability allows neural network computations to be performed efficiently using only the local resources available at each memory location, thereby achieving complex neural network functionality with minimal energy loss.
Data Source
AI summary
An example device includes a plurality of computational memory banks. Each computational memory bank of the plurality of computational memory banks includes an array of memory units and a plurality of processing elements connected to the array of memory units. The device further includes a plurality of single instruction, multiple data (SIMD) controllers. Each SIMD controller of the plurality of SIMD controllers is contained within at least one computational memory bank of the plurality of computational memory banks. Each SIMD controller is to provide instructions to the at least one computational memory bank.