PIM Device MAC Operator Neural Network Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current processing-in-memory (PIM) devices face limitations in performing deterministic arithmetic operations efficiently due to the separation of processors and memory, leading to degraded performance in artificial intelligence applications, particularly in deep learning processes where increased computational demands are exponential and data communication limitations hinder performance.
Innovation Solution
A PIM device is designed with multiple memory banks and a multiplication-accumulative addition (MAC) operator, allowing for deterministic arithmetic operations by writing and reading data from memory banks and performing calculations within the device, with a column control circuit generating signals for arithmetic operations and controlling data access, enabling efficient neural network computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a general hardware system with separated memory and processor is used, then the system structure is simple, but the performance of artificial intelligence is degraded due to data communication limitations
Solution Approach 1:
The patent merges the processor and memory into a single integrated device, combining computational units with storage units in the same semiconductor chip. This eliminates the need for data communication between separate components, thereby improving AI performance while maintaining reasonable system structure complexity.
Solution Approach 2:
The integrated PIM device performs multiple functions by combining both storage and computation capabilities in one unit. The device can store data in memory banks and perform arithmetic operations using MAC operators, making it a universal component that handles both data retention and processing tasks.
2Productivity
If the amount of data communication between memory and processor is increased to improve AI performance, then the computational demand is met, but the data communication limitation hinders performance
Solution Approach 1:
By integrating the processor within the memory chip, the patent eliminates data communication bottlenecks. The MAC operators are directly coupled to the memory banks, allowing computational units to access stored data without external communication, thus maintaining high processing speed while meeting computational demands.
3Ease of operation
If arithmetic operations are performed in a separate processor, then the computational logic is clear, but the data processing speed in neural networks is reduced
Solution Approach 1:
The patent combines arithmetic operation capabilities directly within the memory device by integrating MAC operators with memory banks. This allows neural network computations to be performed at the location where data is stored, dramatically increasing data processing speed while maintaining clear computational logic through dedicated arithmetic units.
Data Source
AI summary
A PIM device writes elements of a first matrix to a first memory bank, and may writes elements of a second matrix to a second memory bank. The PIM device simultaneously reads elements with the same order among the elements of the first and second matrices by simultaneously accessing the first and second memory banks. An MAC operator generates arithmetic data by performing a calculation on data that is read from the first and second memory banks, and writes the arithmetic data to a third memory bank.