In-Memory Computing Memory Device for Parallel MAC Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI systems face an IO bottleneck and inefficient MAC operation flow due to large amounts of input data and weights, leading to reduced performance and accuracy.
Innovation Solution
An In-Memory-Computing memory device with multiple page buffers and memory planes that perform parallel bit multiplication and accumulation, reducing the need for complex arithmetic logic units and enhancing parallelism in MAC operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional AI structure with complex arithmetic logic units is used to perform MAC operations, then computation accuracy can be maintained, but IO bottleneck worsens and operation efficiency decreases
Solution Approach 1:
The patent replaces traditional mechanical arithmetic logic units with an in-memory computing system that performs MAC operations directly within the memory array. Memory cells store weights and perform multiplication through conductance multiplication, with accumulation done through parallel read operations, eliminating the need for separate ALU components and data movement between memory and processor.
Solution Approach 2:
The memory array serves multiple functions simultaneously: it acts as both storage medium for weights and computation unit for MAC operations. The same memory cells that store data can perform multiplication and accumulation, eliminating the distinction between storage and processing functions.
2Measurement precision
If multi-bit inputs and multi-bit weights are used to achieve high accuracy MAC operations, then computation precision improves, but IO bottleneck becomes worse and efficiency decreases
Solution Approach 1:
The patent segments multi-bit MAC operations into multiple single-bit operations. Each bit of the input vector and weight vector is processed separately through sequential single-bit MAC operations, with results accumulated to achieve the final multi-bit precision result. This segmentation allows parallel processing of multiple bit pairs simultaneously.
Solution Approach 2:
The patent transforms the problem from processing multi-bit values in a single dimension to processing multiple single-bit values in parallel across different dimensions (memory planes and page buffers). By distributing bit pairs across multiple memory planes that can operate simultaneously, the system achieves high precision without sequential processing delays.
3Productivity
If more memory cells are used to store weights and perform parallel MAC operations, then computation speed and parallelism improve, but device complexity and operation costs increase
Solution Approach 1:
The patent merges multiple memory planes and page buffers into a unified in-memory computing system. The memory planes are coupled together with shared control logic, allowing coordinated parallel operations. The accumulation circuits aggregate results from multiple planes, effectively combining their computational power while managing complexity through hierarchical organization.
Solution Approach 2:
The system dynamically configures which memory planes and page buffers are active for each MAC operation based on the input data and weight distribution. The control logic adapts the parallelism level and resource allocation dynamically, enabling efficient utilization of memory cells without requiring all cells to be permanently configured for computation.
Data Source
AI summary
A memory device and an operation method thereof are provided. The memory device includes: a plurality of page buffers, storing an input data; a plurality of memory planes coupled to the page buffers, based on received addresses of the memory planes, a plurality of weights stored in the memory planes, the memory planes performing bit multiplication on the weights and the input data in the page buffers in parallel to generate a plurality of bit multiplication results in parallel, the bit multiplication results stored back to the page buffers; and at least one accumulation circuit coupled to the page buffers, for performing bit accumulation on the bit multiplication results of the memory planes in parallel or in sequential to generate a multiply-accumulate (MAC) operation result.