Processing-in-Memory Block for Neural Network Data Movement Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing systems face inefficiencies in processing neural networks due to high power consumption and the need for data movement between memory and processing units, which limits performance.
Innovation Solution
A memory device with a processing-in-memory (PIM) block that includes a control circuit, ALU, and register files, enabling operations directly within the memory architecture, reducing the need for data transfer and enhancing efficiency and power management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a specialized hardware accelerator is implemented for neural network processing, then processing performance is improved, but device complexity increases
Solution Approach 1:
The patent combines the processing unit with the memory device to create an integrated processing-in-memory system. The processing unit includes multiple processing elements that can perform neural network operations directly within the memory device, merging storage and processing functions into a single integrated component. This reduces the need for separate hardware accelerators while maintaining high processing performance for neural network workloads.
2Productivity
If data is transferred between memory and processing unit, then processing capability is improved, but power consumption increases
Solution Approach 1:
The processing unit is integrated within the memory device structure, allowing processing elements to access stored data without external data transfer. This internal integration eliminates the energy-consuming data movement between separate memory and processing components, while still providing full neural network processing capability through the embedded processing elements.
3Reliability
If multiple operation commands are processed sequentially with fence operations, then data integrity is maintained, but processing speed decreases
Solution Approach 1:
The processing elements maintain status information and buffer data internally, performing preliminary preparation for subsequent operations. This allows the system to process multiple operation commands in parallel without requiring fence operations between each command, as the processing elements are already prepared with the necessary data and state information, thus maintaining data integrity while significantly improving processing speed.
Solution Approach 2:
The processing elements can continuously process operation commands without interruption from fence operations. By maintaining internal buffers and status information, the system ensures continuous useful action is performed, eliminating the idle periods caused by sequential fence operations while still ensuring data integrity through proper buffer management and status tracking.
Data Source
AI summary
A memory device includes a first scalar register file storing a first input fragment, a second scalar register file storing a second input fragment, an arithmetic logic unit (ALU), and a control circuit. The control circuit is configured to perform, using the ALU, a first operation between the first input fragment and a first weight fragment based on a first operation command received from a host, and to perform, using the ALU, a second operation between the second input fragment and a second weight fragment based on a second operation command received from the host.