In-Memory LUT MAC Architecture for Low-Energy Neural Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing in-memory processing techniques for machine learning models, such as neural networks, face challenges in achieving optimal energy efficiency and memory density due to the need for extensive data movement and modifications to memory bit-cells, which limits scalability and increases energy consumption, especially for deep neural network workloads.
Innovation Solution
A highly optimized lookup table (LUT)-based approach is implemented within the memory to perform deep processing-in-memory operations without altering bit-cells or peripheral circuitry, reducing data movement and energy consumption by integrating compute logic directly in memory, enabling parallel processing without significant area or power penalties.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If in-memory processing is implemented using traditional techniques, then compute functionality is added to memory, but energy consumption increases due to extensive data movement
Solution Approach 1:
The patent merges compute logic (multiplexer, adder, shifter) directly within the memory array structure, combining storage and processing functions into a unified in-memory compute unit. This integration eliminates data movement between separate memory and compute units, reducing energy consumption while maintaining compute capability.
2Productivity
If in-memory processing is implemented by modifying bit-cells, then compute operations can be performed in memory, but memory density decreases and scalability is limited
Solution Approach 1:
The patent segments the memory array into distinct regions: storage cells for data retention and compute logic units (multiplexer, adder, shifter) for processing operations. This segmentation allows compute functionality to be added without modifying the fundamental bit-cell structure, preserving memory density while enabling in-memory processing.
3Productivity
If extensive data movement is used for machine learning operations, then computation can be performed, but energy consumption increases
Solution Approach 1:
The patent implements self-service by performing multiply and accumulate operations directly within the memory array using integrated compute logic. Data remains stationary in the memory array throughout the computation process, with all operations (multiplication via LUT, accumulation via adder) completed in-place, eliminating energy-consuming data transfers to external compute units.
4Quantity of substance
If compute logic is integrated in memory without modifying bit-cells, then memory density is preserved, but implementing MAC operations becomes challenging
Solution Approach 1:
The patent introduces a lookup table (LUT) as an intermediary mechanism to simplify multiplication operations. The LUT stores pre-computed multiplication results, allowing the multiplexer to retrieve product values directly without complex multiplication circuitry. This intermediary approach enables MAC operations to be implemented using simple memory lookups and additions, reducing compute logic complexity while preserving memory density.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture are disclosed to improve in-memory multiply and accumulate operations. An example apparatus includes a first multiplexer in a subarray of memory, the first multiplexer to receive first values representative of a column of a lookup table (LUT) including entries to represent products of four-bit numbers and return second values from an intersection of a row and the column of the LUT based on a first element of a first operand; shift and adder logic in the subarray, the shift and adder logic to shift the second values based on at least one of the first element of the first operand or a first element of a second operand; and accumulation storage in the subarray, the accumulation storage to store at least the shifted second values.