In-Memory Binary Matrix Multiplication With Majority Sense Amplifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional matrix multiplication in neural networks requires multiple clock cycles and significant data movement between memory and processing units, leading to latency and energy inefficiencies, particularly in large and complex networks.
Innovation Solution
Implementing a single cycle binary matrix multiplication using an associative processing unit (APU) with a memory array, input unit, and majority sense amplifiers, where binary weights and activations are stored and operations are performed in-memory, utilizing NXOR operations and majority functions to achieve efficient binary multiplication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional multiply-accumulate architecture is used with separate memory and processing units, then the system can perform floating point operations, but data movement between memory and processing units increases latency and energy consumption
Solution Approach 1:
The patent merges memory and processing units into a single integrated structure where weight data is stored in memory cells that directly perform multiplication and accumulation operations. This eliminates the need for separate data movement between memory and processing units, reducing energy consumption while maintaining computational capability.
Solution Approach 2:
The patent introduces sense amplifiers as intermediary components that directly read and process data from memory cells during computation. These sense amplifiers serve as mediators between storage and processing functions, enabling in-memory computation without traditional data movement paths.
2Productivity
If conventional multi-step process with separate memory loading is used, then weights can be stored in memory, but multiple clock cycles are required increasing latency
Solution Approach 1:
The patent enables continuous computation by performing multiplication and accumulation operations directly within the memory array during the same clock cycle that weight data is accessed. This eliminates idle cycles between data loading and computation, maintaining continuous useful action and reducing overall latency.
Solution Approach 2:
The patent pre-positions weight data in memory cells in a format that is immediately ready for computation. The memory array is configured and initialized beforehand so that when computation is needed, the data is already in the correct location and format, eliminating preliminary data movement steps.
3Measurement precision
If floating point weights are stored and processed, then precision is maintained, but significant data movement between memory and processing units is required
Solution Approach 1:
The patent replaces traditional floating-point arithmetic operations with boolean logic operations (AND, OR, NOT) performed directly in the memory array. This substitution of computational mechanics eliminates the need for complex data movement associated with floating-point operations while maintaining sufficient precision for neural network applications.
Data Source
AI summary
A system and method for single cycle binary matrix multiplication in neural network computations is disclosed. The system includes a memory array storing binary weights, an input unit for activating rows based on a binary activation vector, and per-column majority sense amplifiers. The system performs binary matrix multiplication in a single cycle, enabling efficient implementation of binary neural networks. The memory array may include sections for weights and inverse weights, with corresponding activation register sections. Differential sense amplifiers may implement the majority function. The system can be applied to convolutional neural networks, using SRAM arrays for image storage and processing. Methods for determining majority votes and counting activated bits using iterative modification of the activation vector are also described.