PIM Device Selective MAC Operations for AI Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing complexity of neural networks in AI systems leads to exponential computation requirements, which are hindered by the limitations of data communication between separate processor and memory units in traditional hardware systems, resulting in degraded AI performance.
Innovation Solution
A Processing-in-Memory (PIM) device is proposed, integrating processor and memory on a semiconductor chip, featuring multiple storage regions, global buffers, and multiplication and accumulation (MAC) circuits, allowing for selective MAC operations by active or inactive MAC circuits to optimize data processing and reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a general hardware system with separate memory and processor is used, then the system structure is simple and easy to manufacture, but the AI performance is degraded due to data communication limitations between memory and processor
Solution Approach 1:
The patent merges the processor and memory into a single integrated PIM device, where MAC circuits are directly coupled to storage regions within the same semiconductor chip. This integration eliminates the need for separate memory and processor units, resolving the contradiction by combining previously separate components to achieve both improved AI performance through reduced data communication limitations and a unified structure that maintains manufacturing feasibility
2Productivity
If the number of layers in neural network is increased to improve AI performance, then the computation capability is enhanced, but the amount of computation required increases exponentially
Solution Approach 1:
The patent segments the computation process into distributed MAC operations performed by multiple MAC circuits that are locally coupled to storage regions. Each MAC circuit independently performs multiplication and accumulation operations on data from its associated storage regions, enabling parallel processing that enhances computation capability for deep neural networks while distributing energy consumption across multiple independent units rather than concentrating it in a single processor
Solution Approach 2:
The patent transitions from sequential processing in traditional architectures to a two-dimensional array architecture where MAC circuits are arranged in rows and columns, with storage regions similarly organized. This spatial dimensionality enables massive parallelism, allowing the system to handle the exponentially increasing computation requirements of deep neural networks by processing multiple operations simultaneously across the array, thereby improving AI performance without proportionally increasing energy consumption
3Speed
If all MAC circuits are activated to perform MAC operations, then the processing speed is improved, but the power consumption increases
Solution Approach 1:
The patent implements dynamic control of MAC circuits through enable signals that can selectively activate or deactivate individual MAC circuits based on computational requirements. This dynamic switching capability allows the system to activate only the necessary number of MAC circuits for each operation, maintaining high processing speed when needed while reducing power consumption during operations requiring fewer computational units, thus resolving the contradiction between speed and energy usage
Data Source
AI summary
A processing-in-memory (PIM) device includes a plurality of storage regions, a global buffer, and a plurality of multiplication/accumulation (MAC) circuits. The plurality of MAC circuits are configured to perform a MAC operation of first data from the plurality of storage regions and second data from the global buffer. Each of the plurality of MAC circuits is categorized as either an active MAC circuit or an inactive MAC circuit. The MAC operation includes a selective MAC operation which is selectively performed by the active MAC circuit.