Host Expansion Control Circuit for Near Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As the number of parameters in deep neural networks increases, the computational performance of accelerators for operations such as matrix multiplication is hindered by inefficient memory and communication operations, leading to increased time spent in these operations.
Innovation Solution
A host device with a unit processor, a host expansion control circuit, and a local memory device is implemented to perform near data processing (NDP) requests, allowing for memory operations and computation operations to be performed concurrently on the local memory device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If computation circuits are added inside the memory device to enable near data processing, then computational performance is improved, but storage space is sacrificed
Solution Approach 1:
The system divides computational tasks into segments: the host device's unit processor generates NDP requests for specific computation operations, while the host expansion control circuit manages memory operations. This segmentation allows computation and memory access to be separated and executed concurrently without requiring computation circuits inside the memory device, thus maintaining full storage capacity while improving computational performance.
Solution Approach 2:
The host expansion control circuit acts as an intermediary between the unit processor and the local memory device. It receives NDP requests from the unit processor and coordinates memory read/write operations with computation operations, enabling concurrent execution without modifying the memory device structure. This intermediary approach resolves the contradiction by enabling near data processing functionality without sacrificing storage space.
2Duration of action of moving object
If memory operations and computation operations are performed sequentially, then device complexity is reduced, but operation time increases
Solution Approach 1:
The host expansion control circuit enables continuous useful action by overlapping memory read/write operations with computation operations. When the unit processor issues an NDP request, the control circuit initiates memory operations and computation operations simultaneously, ensuring that neither operation waits for the other. This continuous execution reduces total operation time without significantly increasing device complexity, as the control circuit already manages memory operations.
Solution Approach 2:
The system performs preliminary action by having the unit processor generate NDP requests in advance, which are then processed by the host expansion control circuit to pre-coordinate memory and computation operations. This preliminary preparation enables concurrent execution to begin immediately when data is ready, reducing overall operation time without requiring complex real-time coordination during execution.
3Productivity
If the proportion of time spent in memory operations increases, then computational performance appears improved, but overall efficiency decreases
Solution Approach 1:
The host expansion control circuit implements periodic action by alternating between memory operations and computation operations in a coordinated manner. When one operation completes, the next operation begins immediately, creating a periodic rhythm of execution that keeps both memory and computation units continuously utilized. This periodic execution pattern reduces idle time and improves overall efficiency while maintaining high computational performance.
Data Source
AI summary
A host device includes a unit processor configured to generate a near data processing (NDP) request, a host expansion control circuit configured to receive the NDP request; and a local memory device configured to store data corresponding to the NDP request according to control by the expansion control circuit. In response to receiving the NDP request, the host expansion control circuit performs a request processing operation to perform a read or a write operation corresponding to the NDP request on the local memory device and performs a computation operation using the requested data corresponding to the NDP request.