In-Memory Multiply-Accumulate via Global Charge-Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face significant bottlenecks in artificial intelligence computations due to the high cost of storing large neural network data in processor caches, leading to inefficient data transfer and increased energy consumption, as machine learning operations require numerous data elements that are typically stored in memory rather than cache.
Innovation Solution
The implementation of compute-in-memory (CIM) circuits that perform multiply-accumulate (MAC) operations within memory using shared charge, allowing for local processing of dot-products and absolute differences of vectors without transferring data to the host processor, thereby reducing data movement and energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in processor cache to enable fast computation, then computation speed is improved, but memory cost and energy consumption increase due to the large amount of data required for neural network operations
Solution Approach 1:
The patent merges memory and computation functions into a single compute-in-memory circuit. The memory array performs both data storage and MAC operations simultaneously, eliminating the need to transfer data between separate memory and processor components. This integration reduces energy consumption associated with data movement while maintaining fast computation speeds.
Solution Approach 2:
The memory circuit performs computation operations autonomously without requiring data to be transferred to an external processor. The MAC operations are executed directly within the memory array using the stored data, allowing the system to self-service computational tasks and eliminate energy-wasting data transfers.
2Productivity
If data is transferred between memory and processor, then computation can be performed, but data transfer time and energy consumption increase significantly
Solution Approach 1:
By combining memory and computation functions into a single integrated circuit, the patent eliminates the physical data transfer step between separate memory and processor components. MAC operations are performed in-place within the memory array, drastically reducing data transfer time and increasing overall computation throughput.
Solution Approach 2:
The patent extracts the computation function from the traditional processor and places it directly within the memory circuit. This extraction eliminates the data transfer bottleneck by allowing computation to occur where the data is already stored, significantly reducing transfer time and improving productivity.
3Adaptability or versatility
If large amounts of data are stored in processor cache to support machine learning operations, then computation capability is improved, but system cost increases prohibitively
Solution Approach 1:
The compute-in-memory circuit serves multiple functions: it acts as both memory storage and computation unit. This multi-functionality allows the same hardware structure to handle both data storage and MAC operations, reducing the need for separate high-cost cache memory and processing units, thereby lowering overall system cost while maintaining computation capability.
Solution Approach 2:
By merging memory and processor functions into a single integrated circuit, the patent reduces system complexity and cost. The unified structure eliminates the need for separate high-capacity cache memory and powerful processors, achieving the same computational capability with lower overall system cost.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the throughput and performance of machine learning operations by performing MAC operations within the memory array, reducing data transfer and energy consumption, and enabling higher throughput and lower energy usage compared to traditional host processor computations.
Implementation Method 1
Column access circuitry to drive a column of the memory array when the rows are driven. Charge accumulates on the column line. Sensing circuitry coupled to the column, to sense voltage on the column
Data Source
AI summary
A memory circuit has compute-in-memory circuitry that enables a multiply-accumulate (MAC) operation based on shared charge. Row access circuitry drives multiple rows of a memory array to multiply a first data word with a second data word stored in the memory array. The row access circuitry drives the multiple rows based on the bit pattern of the first data word. Column access circuitry drives a column of the memory array when the rows are driven. Accessed rows discharge the column line in an accumulative fashion. Sensing circuitry can sense voltage on the column line. A processor in the memory circuit computes a MAC value based on the voltage sensed on the column.