In-Memory Processing Engine for Parallel Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Von Neumann computing architecture experiences inefficiencies and memory bottlenecks when handling large data sets with low locality, such as those used in deep learning and artificial neural networks, due to inefficient cache memory usage and high power consumption in simultaneous operations across all memory banks.
Innovation Solution
A memory device performs in-memory operations by using an operand shared by all banks, with a Processing In-Memory Interface Unit (PIM IU) that delivers operation requests and a PIM engine in each bank to read and process data, allowing for efficient matrix multiplication across multiple memory banks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the per-bank method is used for in-memory operations, then the memory device can perform operations in each bank, but the execution time becomes longer because operations of each bank are executed sequentially
Solution Approach 1:
The patent segments the operand data into multiple units and distributes them across different memory banks. Each bank processes a specific segment independently, allowing parallel execution of operations across banks rather than sequential processing, thereby reducing total execution time while maintaining manageable complexity in each individual bank.
Solution Approach 2:
The patent merges the processing capabilities of multiple memory banks by having them operate simultaneously on different segments of the same operation. The PIM controller coordinates these parallel operations and combines the results, achieving faster overall execution compared to sequential per-bank processing.
2Productivity
If the all-bank method is used for in-memory operations, then all banks can execute operations simultaneously, but power consumption becomes high and a memory controller supporting simultaneous operations of all banks is required
Solution Approach 1:
The patent applies local quality by enabling operations only in the specific memory banks that contain the required data segments, rather than forcing all banks to operate simultaneously. This selective activation reduces power consumption while maintaining parallel processing benefits for the necessary banks.
Solution Approach 2:
The patent implements partial action by activating only the subset of memory banks needed for the current operation based on data locality requirements, rather than activating all banks. This reduces unnecessary power consumption from idle banks while still achieving parallel processing acceleration.
3Speed
If data with low locality is stored in cache memory for quick access, then the cache memory structure is utilized, but the cache memory is not used efficiently and memory bottlenecks occur due to large data capacity requirements
Solution Approach 1:
The patent introduces PIM engines as intermediary processing units embedded within the memory device. These engines perform operations directly on data stored in memory banks, acting as a mediator between the cache memory and main processing units. This eliminates the need to transfer large datasets between cache and memory, improving both access speed and cache efficiency.
Solution Approach 2:
The patent adds a new dimension of computation by embedding processing capabilities directly within the memory hierarchy. Instead of the traditional sequential model (memory → cache → processor), the system enables in-memory computation that operates parallel to data transfer, effectively adding a spatial dimension to data processing that bypasses cache memory bottlenecks.
Data Source
AI summary
Disclosed is a memory device including a plurality of memory banks, each of which performs an operation based on first operand data including pieces of first unit data and second operand data including pieces of second unit data and a processing in-memory interface unit (PIM IU) that delivers signals for an operation request to the plurality of memory banks. Each of the plurality of memory banks includes a memory cell array configured to store one of the pieces of first unit data and a PIM engine that reads the one of the pieces of first unit data from the memory cell array, reads the pieces of second unit data broadcast to the plurality of memory banks, and generates an operation result by performing an operation based on the one of the pieces of first unit data and the pieces of second unit data.