Banked Memory Device With Process Units for Memory-Wall Relief
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The memory bottleneck in high-performance computing systems, particularly in AI systems using transformer models, limits performance due to the lag in memory access speed compared to processor computation speed, creating a constraint known as the 'memory wall'.
Innovation Solution
A memory device with banks of memory cells and a peripheral circuit that includes a control logic for programming data into different banks and process units to perform calculations, distributing calculation tasks within the memory device, thereby reducing the need for large data transfers to the processor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If memory access speed is increased to match processor computation speed, then the memory wall constraint is resolved, but device complexity and manufacturing difficulty increase significantly
Solution Approach 1:
The patent merges the memory function and computation function into a single memory device. The memory device includes memory cells for data storage and process units for performing calculations, allowing computation to be executed directly within the memory device without requiring separate processor components, thus resolving the memory wall constraint while avoiding the complexity of a separate high-speed memory system
Solution Approach 2:
The memory device is designed with multi-functionality, serving both as a data storage medium and a computation unit. The process units within the memory device can perform calculations on stored data, enabling the memory device to handle both memory access and computation tasks, thereby eliminating the need for separate high-speed memory and processor components
2Productivity
If data is distributed across multiple banks of memory cells, then calculation throughput is improved, but device complexity and control difficulty increase
Solution Approach 1:
The memory device is segmented into multiple banks of memory cells, with each bank capable of storing data independently. Process units are distributed across these banks to perform calculations on locally stored data, enabling parallel processing and improved throughput while maintaining manageable control through the peripheral circuit
Solution Approach 2:
The patent introduces a spatial dimension to data processing by distributing data across multiple banks of memory cells in a two-dimensional array structure. This dimensional organization allows process units to access and process data from different banks simultaneously, improving calculation throughput while the peripheral circuit manages the complexity of coordinating these distributed operations
3Loss of time
If process units are integrated within the memory device, then data transfer to processor is reduced, but manufacturing precision requirements increase
Solution Approach 1:
The patent combines the process units with the memory cells within the same memory device structure. This integration allows process units to directly access and process data stored in adjacent memory cells without requiring data transfer to external processors, significantly reducing data transfer time while the integration is designed to meet standard manufacturing precision capabilities
Data Source
AI summary
A memory device, a memory system, and a method for data calculation with the memory device are provided. The memory device includes an array of memory cells and a peripheral circuit coupled to the memory cells is provided. The peripheral circuit includes a control logic configured to program first data and second data into different banks of the banks of memory cells, at least one process unit configured to perform calculation based on the first data and the second data, and a data-path bus coupled to the control logic and the at least one process unit to transmit the first date and the second data.