Processing-In-Memory MAC Architecture for Faster Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The separation of memory and processor in traditional hardware systems limits data communication, degrading the performance of artificial intelligence due to increased computational demands in neural networks, especially with deep learning applications.
Innovation Solution
Integration of processors and memory in a semiconductor chip, utilizing PIM devices with MAC operators, memory banks, and data I/O circuits to perform arithmetic operations directly, enhancing data processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If memory and processor are separated in traditional hardware systems, then device complexity is reduced and ease of manufacture is improved, but data communication speed is limited and productivity is degraded
Solution Approach 1:
The patent merges memory and processor functions into a single PIM device by integrating MAC operators directly with memory banks. This allows arithmetic operations to be performed at the memory location, eliminating data transfer between separate memory and processor components, thereby improving data processing speed while managing integration complexity through systematic architectural design.
2Productivity
If the number of neural network layers is increased to improve AI performance, then computational capability is enhanced, but the amount of computation required increases exponentially
Solution Approach 1:
The patent segments the computational workload by distributing MAC operators across multiple memory banks, with each bank handling specific portions of matrix operations. This segmentation allows parallel processing of different data subsets, reducing the computational burden on any single unit and enabling efficient handling of deep neural networks with multiple layers.
Solution Approach 2:
The patent enables continuous arithmetic operations by keeping data resident in memory banks during computation. The MAC operators perform calculations directly on stored data without requiring data to be moved to external processors, maintaining continuous useful action and reducing idle time between computational stages in deep learning workloads.
3Speed
If PIM devices perform arithmetic operations directly in memory, then data processing speed is improved, but device complexity increases due to integration of MAC operators and memory banks
Solution Approach 1:
The patent designs memory banks to serve dual functions: storing data and performing arithmetic operations through integrated MAC operators. This multi-functionality reduces the need for separate processing units and data transfer mechanisms, improving speed while managing complexity by consolidating functions into unified memory structures.
Solution Approach 2:
The patent adds the computational dimension to traditional memory structures by integrating MAC operators within memory banks. This transforms memory from a passive storage component to an active processing element, enabling arithmetic operations to occur in the memory domain rather than requiring data movement to external processors.
Data Source
AI summary
A processing-in-memory (PIM) device includes a plurality of multiplication/accumulation (MAC) operators, a plurality of memory banks, and a plurality of data input/output (I/O) circuits. The plurality of MAC operators is configured to perform a MAC arithmetic operation using weight data of a weight matrix provided by the plurality of memory banks as input data to generate plural sets of MAC result data corresponding to elements of a result matrix. The PIM device is configured to store the weight data of the weight matrix in the memory banks in units of rows of the weight matrix. The PIM device is also configured to sequentially output the plural sets of MAC result data from the PIM device through the plurality of data I/O circuits according to a sequence in which the row number of the result matrix increases.