Processing-in-Memory MAC Architecture for Lower AI Data Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The separation of processor and memory in traditional hardware systems limits data communication, degrading the performance of artificial intelligence due to increased computational demands, especially in deep learning applications.
Innovation Solution
Integration of processor and memory in a semiconductor chip, utilizing MAC operators, memory banks, additional adders, and data I/O circuits to perform deterministic arithmetic operations directly in the PIM device, enhancing data processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If processor and memory are separated in traditional hardware systems, then device complexity is reduced and manufacturing is easier, but data communication between memory and processor is limited, degrading AI performance
Solution Approach 1:
The patent merges the processor and memory into a single integrated PIM device, where MAC operators are directly coupled to memory banks within the same semiconductor chip. This integration eliminates the need for separate processor and memory components, allowing arithmetic operations to be performed directly where data is stored, thereby improving data processing speed while managing device complexity through unified architecture design
2Loss of time
If processor and memory are integrated in PIM device, then data processing speed is improved by performing arithmetic operations directly in memory, but device complexity increases
Solution Approach 1:
The patent integrates MAC operators directly within the memory device, merging computation and storage functions into a single unit. This eliminates data communication latency between separate processor and memory components by performing arithmetic operations directly in the memory device, while managing complexity through systematic organization of memory banks and operators
Solution Approach 2:
The patent segments the integrated PIM device into multiple memory banks (first memory bank, second memory bank, etc.) and corresponding MAC operators, with each bank-operator pair handling specific data processing tasks. This segmentation allows parallel processing operations while maintaining manageable complexity within each segment, and the systematic division enables efficient resource utilization across the integrated device
3Productivity
If multiple memory banks and MAC operators are integrated in PIM device, then data processing throughput is improved, but manufacturing precision requirements increase
Solution Approach 1:
The patent divides the PIM device into multiple independent memory banks and MAC operator units, where each bank-operator pair can function as a modular processing element. This segmentation enables parallel computational operations across multiple units, increasing overall throughput while allowing each individual unit to be manufactured with standard precision requirements, rather than requiring the entire complex system to achieve high precision simultaneously
Data Source
AI summary
A processing-in-memory (PIM) device includes first to Lth multiplication/accumulation (MAC) operators, first to Lth memory banks, first to Lth additional adders, and a plurality of data input/output (I/O) circuits. The first to Lth MAC operators include first to Lth left MAC operators and first to Lth right MAC operators. The first to Lth additional adders are configured to generate and output first to Mth MAC result data. The plurality of data I/O circuits include left data I/O circuits and right data I/O circuits. Each of the first to Lth additional adders is classified as either a left additional adder or a right additional adder. The left additional adders are configured to transmit a first portion of the first to Mth MAC result data to the left data I/O circuits, and the right additional adders are configured to transmit a second portion of the first to Mth MAC result data to the right data I/O circuits.