In-Memory Processing Units with Distance-Aware Memory Bank Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory devices face challenges in efficiently interfacing and processing compute-intensive operations like graphics algorithms and neural networks due to the addition of separate processing hardware, which requires improved methods for efficient operation and interfacing.
Innovation Solution
A memory device with in-memory operation units configured in multi-pipeline stages, where memory banks are assigned to these units, and a bank selector and multiplexer are used to manage data access based on transmission distances, allowing for pipelined in-memory processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate processing hardware is added to the memory device for in-memory processing, then computation capability is improved, but device complexity increases
Solution Approach 1:
The patent combines memory storage and processing functions into a unified memory device structure. The in-memory operation units are integrated directly with the memory banks on the same die, merging what were previously separate components (memory and processing hardware) into a single integrated device, thereby improving computation capability while managing device complexity through consolidation
Solution Approach 2:
The memory device is designed to perform multiple functions: it can operate as a conventional memory device and simultaneously perform computation operations. The in-memory operation units enable the memory device to execute both storage and processing tasks, providing multi-functionality that improves productivity without requiring entirely separate specialized hardware
2Area of stationary object
If memory banks are arranged with different data transmission distances to command pads, then area utilization is improved, but access latency varies
Solution Approach 1:
The patent applies local quality by assigning different memory banks to different in-memory operation units based on their spatial characteristics. Memory banks with shorter transmission distances are assigned to certain operation units while those with longer distances are assigned to others, allowing each unit to optimize its access patterns for its specific local characteristics, thereby managing area utilization while addressing latency variations through localized optimization
Solution Approach 2:
The memory device is segmented into multiple in-memory operation units, each associated with specific memory banks. This segmentation allows the system to handle different access latency characteristics by distributing workloads across multiple units, where each unit manages its own assigned memory banks with known transmission distance characteristics, thus managing both area utilization and access latency through division
3Productivity
If in-memory operation units access memory banks with different transmission distances, then processing throughput is improved, but latency differences increase
Solution Approach 1:
The patent implements dynamic operation modes that allow in-memory operation units to adaptively select between different access patterns. The system can dynamically switch between single-memory-bank mode and dual-memory-bank mode depending on the specific operation requirements and latency constraints, enabling flexible throughput optimization while managing latency differences through adaptive behavior rather than fixed configurations
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A memory device includes: in-memory operation units to perform in-memory processing of an operation pipelined in multi-pipeline stages; memory banks assigned to the plurality of in-memory operation units such that a set of n memory banks is assigned to each of the in-memory operation units, each memory bank performing an access operation of data requested by each of the plurality of in-memory operation units while the pipelined operation is performed, wherein n is a natural number; and a memory die in which the in-memory operation units, the memory banks, and command pads configured to receive a command signal from an external source are arranged. Each set of the n memory banks includes a first memory bank having a first data transmission distance to the command pads and a second memory bank having a second data transmission distance to the command pads that is larger than the first data transmission distance.