Multistage Memory Acceleration for PCIe Data Transfer Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing semiconductor memory devices, such as DRAM, face limitations in data transfer speed and efficiency, particularly when handling neural network operations, due to bottlenecks in data transmission and reception, which affect overall system performance.
Innovation Solution
Implementing a multistage acceleration approach within the memory device by using a first accelerator inside the memory package for coarse acceleration to convert sparse data into dense data, and a second accelerator in the device controller for fine acceleration to perform neural network operations, thereby optimizing data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data transmission between host and memory device follows PCIe protocol, then system compatibility is maintained, but data transmission speed is limited and power consumption increases
Solution Approach 1:
The patent merges the memory device and accelerator into a single integrated package, allowing data processing to occur within the memory package itself rather than requiring frequent data transmission between host and memory device via PCIe interface. This integration eliminates the PCIe transmission bottleneck and reduces power consumption associated with high-speed data transmission.
Solution Approach 2:
The patent segments the acceleration function into a separate accelerator component within the memory package, distinct from the main memory devices. This segmentation allows the accelerator to process data locally using alternative protocols while memory devices continue to function independently, resolving the contradiction by enabling fast local processing without compromising system compatibility.
2Productivity
If all acceleration operations are performed in the memory package, then processing speed increases, but device complexity increases
Solution Approach 1:
The acceleration function is segmented into a separate accelerator component within the memory package, distinct from the main memory devices. This segmentation allows the accelerator to handle complex processing tasks while memory devices maintain their simpler structure and function independently, thus increasing processing speed without proportionally increasing overall device complexity.
Solution Approach 2:
The accelerator is designed as a multi-functional component that can perform various acceleration operations (e.g., matrix multiplication, convolution) within the memory package. This universality allows a single additional component to handle multiple processing tasks, increasing productivity without requiring separate specialized components for each function.
3Ease of operation
If frequent data transmission occurs between host and memory device, then data access is simplified, but power consumption increases and processing efficiency decreases
Solution Approach 1:
The system is segmented into host, memory controller, and memory package with accelerator, allowing data access to be simplified at the host interface level while complex processing occurs locally in the accelerator. This segmentation maintains ease of operation for data access while dramatically improving processing efficiency by eliminating the need for frequent data transmission between host and memory device.
Data Source
AI summary
An electronic device is provided. The electronic host includes: a host; a memory package including a plurality of memory devices and a first accelerator circuit configured to receive first data from the plurality of memory devices and perform a coarse acceleration operation based on the first data to obtain second data; and a memory controller including a second accelerator circuit configured to receive the second data from the first accelerator circuit and perform a fine acceleration operation based on a neural network and the second data to obtain an inference result.