PIM MAC Mode Switching for Throughput and Thermal Stability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The separation of memory and processor in existing hardware systems limits the performance of artificial intelligence hardware due to excessive data communication demands, hindering the improvement of computational efficiency in neural networks.
Innovation Solution
A processing-in-memory (PIM) device with integrated MAC units and a control circuit that allows for both all MAC mode operations and dispersion MAC mode operations, optimizing data processing by performing computations directly within the memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If memory and processor are separated in existing hardware systems, then functional modularity is improved, but data communication requirements increase excessively, worsening computational efficiency
Solution Approach 1:
The patent merges the memory and processor functions by integrating MAC operation units directly within the memory device. This allows data to be processed in-place without being transferred to a separate processor, thereby reducing communication overhead while maintaining functional capabilities through the integrated MAC units that can perform multiplication and accumulation operations directly on stored data
2Productivity
If all MAC units perform MAC operations simultaneously in all MAC mode, then computational throughput is improved, but energy consumption and heat generation increase, worsening system stability
Solution Approach 1:
The patent implements dynamic operation modes that allow the system to switch between all-MAC mode (where all MAC units operate simultaneously for maximum throughput) and dispersion-MAC mode (where MAC operations are distributed across different time periods). This dynamic adjustment enables the system to optimize computational throughput when needed while reducing energy consumption and heat generation during other periods, thereby maintaining system stability
Solution Approach 2:
The dispersion-MAC mode utilizes periodic action by distributing MAC operations across different time periods rather than executing them all simultaneously. This temporal distribution reduces peak energy consumption and heat generation, allowing the system to maintain high computational throughput over time while ensuring stability through periodic operation cycles
3Productivity
If data is transferred between memory and processor, then computation can be performed, but data communication time increases, worsening processing speed
Solution Approach 1:
The memory device provides self-service computational capabilities by integrating MAC operation units within the memory array. Data can be processed directly at its storage location without requiring transfer to an external processor, enabling the memory system to serve its own computational needs and eliminating the speed bottleneck associated with data communication
Data Source
AI summary
A processing-in-memory (PIM) device includes a plurality of multiplication and accumulation (MAC) units, each of the MAC units including a memory bank and a MAC operator, and a control circuit configured to control the plurality of MAC units to perform an all MAC mode operation in which MAC operations are performed in all MAC units, among the plurality of MAC units, or a dispersion MAC mode operation in which the MAC operations are performed in some MAC units, among the plurality of MAC units.