Memory LUT Processing Unit for Parallel Low-Latency Table Lookup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory devices face challenges in efficiently performing table lookup operations due to the computational complexity and power consumption associated with complex activation functions, which require significant hardware resources and impact performance.
Innovation Solution
A processing unit for table lookup is implemented with hardware circuitry that utilizes a lookup table (LUT) for fast retrieval of output values, allowing parallel execution and reducing latency and improving throughput by replicating this design across memory banks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complex activation functions are implemented in existing memory devices, then computational capability is improved, but hardware complexity and power consumption increase significantly
Solution Approach 1:
The patent segments the computational task by separating the activation function evaluation from the main memory array. The memory array is divided into banks, with dedicated processing units handling table lookup operations for each bank. This segmentation allows complex computational functionality to be added without overwhelming the entire memory device structure.
Solution Approach 2:
The patent introduces an intermediary processing unit between the memory array and the output that handles complex activation functions. This intermediary contains lookup tables and logic circuits that perform the computationally intensive operations, allowing the main memory array to remain relatively simple while still achieving advanced computational capability.
2Adaptability or versatility
If complex activation functions are implemented in existing memory devices, then computational capability is improved, but power consumption increases significantly
Solution Approach 1:
The patent pre-computes and stores activation function values in lookup tables during manufacturing or initialization. During operation, the processing units simply retrieve pre-computed values from these tables rather than performing complex calculations in real-time, dramatically reducing power consumption during active computational tasks.
Solution Approach 2:
The patent creates simplified copies of activation function logic in the form of lookup tables and combinatorial logic circuits within each processing unit. These copies enable fast, low-power retrieval of activation values without requiring the full computational infrastructure of general-purpose processors.
3Ease of manufacture
If table lookup operations are performed using conventional methods, then implementation is simple, but execution speed and throughput are limited
Solution Approach 1:
The patent merges the table lookup function with the memory array structure itself. Processing units are integrated directly into memory banks, allowing data to be fetched from memory and processed through activation functions in a single unified operation rather than requiring separate memory access and processing stages.
Solution Approach 2:
The patent transitions from sequential table lookup to parallel processing by implementing multiple processing units that can simultaneously handle different memory banks or different elements within banks. This dimensional expansion from single-unit sequential processing to multi-unit parallel processing dramatically increases throughput while maintaining the fundamental table lookup approach.
4Productivity
If parallel execution is implemented across memory banks, then throughput is improved, but device complexity increases
Solution Approach 1:
The patent implements identical processing units across multiple memory banks, where each unit performs the same table lookup and activation function operations. This universal design allows parallel execution across banks while using replicated, standardized logic blocks rather than requiring complex custom circuitry for each bank, managing complexity through regularity and repetition.
Data Source
AI summary
A processing unit of memory for table lookup is described herein. A plurality of elements (e.g., output values) of a lookup table (LUT) can be sequentially prefetched from a respective column of memory cells that is indicated by each vector value of vector values stored in positions of a register of the processing unit. Each of the vector values can be shifted by one position among the positions of the register to cause a terminal position of the register to be available for storing the respective output value among the prefetched output values.