Memory Expansion Device Near Data Processing Control Circuit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computational performance of accelerators for deep neural networks is hindered by inefficient memory operations and communication operations, despite advancements in arithmetic operation performance. Additionally, near data processing (NDP) and processing in memory (PIM) technologies sacrifice storage space by integrating computation circuits within memory devices.
Innovation Solution
A memory expansion device with an expansion control circuit that receives NDP requests and controls a remote memory device to perform memory operations and computation operations. This system includes an interconnect circuit connecting host devices, memory expansion devices, and allows for on-the-fly NDP operations, reducing the need for frequent data transfer between the GPU and local memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If computation circuits are integrated inside the memory device to perform near data processing, then computational performance is improved, but storage space is sacrificed
Solution Approach 1:
The system divides the near data processing function into two segments: the expansion control circuit in the host device that performs computation operations, and the remote memory device that provides storage. This segmentation allows computation and storage to be separated spatially while maintaining functional integration, thus improving computational performance without sacrificing storage space.
Solution Approach 2:
The interconnect circuit acts as an intermediary between the host device and remote memory device, enabling efficient communication and data transfer. This intermediary infrastructure allows the system to achieve near data processing performance without requiring physical integration of computation circuits within the memory device, thereby preserving storage capacity.
2Productivity
If memory operation and communication operation performance are improved, then data transfer efficiency is enhanced, but the proportion of time spent in these operations increases due to lag compared to computation operation improvements
Solution Approach 1:
The system performs computation operations on data while it is being transferred between the host device and remote memory device. By initiating computation operations in advance during data transfer, the system overlaps computation time with data transfer time, thereby reducing the total time spent on memory operations while maintaining improved data transfer efficiency.
Solution Approach 2:
The system maintains continuous useful action by overlapping memory read/write operations with computation operations. The expansion control circuit continuously processes data during transfer, ensuring that computation resources remain utilized throughout the data transfer process rather than remaining idle, thus reducing overall operation time while maintaining high data transfer efficiency.
Data Source
AI summary
A memory expansion device includes an expansion control circuit configured to receive a near data processing (NDP) request and a remote memory device configured to store data corresponding to the NDP request according to control by the expansion control circuit. In response to the NDP request, the expansion control circuit performs a request processing operation to perform a read or a write operation on the remote memory device and performs a computation operation using the data corresponding to the NDP request.