Compute-In-Memory Circuitry for Off-Chip Bandwidth Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Bandwidth and latency constraints in transferring data between off-chip memory and programmable logic devices for arithmetic computations in AI and machine learning tasks hinder efficient performance.
Innovation Solution
Implementing compute-in-memory circuitry within the off-chip memory, allowing arithmetic computations to take place in the same die as the data storage, thereby reducing data transfer requirements and enhancing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in off-chip memory and transferred to programmable logic device for computation, then data storage capacity is improved, but bandwidth and latency constraints worsen computation efficiency
Solution Approach 1:
The patent merges memory storage and computation functions into a single integrated circuit die. The memory array and compute units are fabricated together on the same die, allowing data to remain in memory while computation occurs directly on the stored data. This eliminates the need to transfer data between separate memory and processor components, thereby resolving the bandwidth and latency constraints that hinder computation efficiency while maintaining large data storage capacity.
2Quantity of substance
If data is transferred from off-chip memory to accelerator for computation, then data storage is improved, but data transfer bandwidth constraints worsen system performance
Solution Approach 1:
The patent combines memory and computation resources on a single integrated circuit die, eliminating the physical separation between storage and processing components. This integration removes the data transfer interface entirely, as data remains in the memory array during computation operations. Consequently, the bandwidth constraints associated with data transfer between separate memory and accelerator components are completely eliminated, while data storage capacity is maintained through the integrated memory array.
3Quantity of substance
If data is transferred from off-chip memory to accelerator, then data storage capacity is improved, but latency constraints worsen computation speed
Solution Approach 1:
The patent integrates memory storage and computation units on the same integrated circuit die, creating a unified architecture where data resides in memory and computation occurs in-place. This eliminates the latency associated with data transfer between separate memory and processor components. The compute units access data directly from the integrated memory array without external transfers, thereby minimizing computation latency while maintaining substantial data storage capacity through the on-die memory.
Data Source
AI summary
An integrated circuit device may include programmable logic circuitry on a first integrated circuit die and memory that includes compute-in-memory circuitry on a second die. The programmable logic circuitry may be programmed with a circuit design that operates on a first set of data. The compute-in-memory circuitry of the memory may perform an arithmetic operation using the first set of data from the programmable logic circuitry and a second set of data stored in the memory.