In-Memory Computing Memory Device Bitwise MAC Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI systems face an IO bottleneck and inefficient MAC operations due to large power consumption and chip size issues in NVM-based In-Memory-Computing, particularly with multi-bit inputs and weights, leading to reduced efficiency and accuracy.
Innovation Solution
A memory device with a memory array for storing weights and a multiplication circuit performing bitwise multiplication, followed by bitwise counting to generate MAC operation results, utilizing a unique data mapping technique that reduces memory cell usage and power consumption through unary folding data product (unFDP) format, which decreases memory cell requirements and improves error correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If non-volatile In-Memory-Computing is used to accelerate MAC operations, then computational efficiency is improved, but power consumption increases due to large summation current after weighted
Solution Approach 1:
The patent segments the weight values into multiple bits (e.g., 2-bit weights) and performs separate bitwise multiplication operations for each bit combination. This segmentation allows the system to use smaller current levels for each operation, reducing the total summation current and power consumption while maintaining computational accuracy through multiple passes.
Solution Approach 2:
The patent performs multiple partial MAC operations (e.g., four separate operations for 2-bit weights) instead of one complete operation. Each partial operation processes a subset of weight bits, allowing the current to be reset and reused across multiple cycles, thereby reducing the peak current requirement and overall power consumption.
2Productivity
If non-volatile In-Memory-Computing is used to accelerate MAC operations, then computational efficiency is improved, but chip size increases due to large number of ADC/DAC components
Solution Approach 1:
The patent replaces complex ADC/DAC conversion mechanisms with direct bitwise multiplication and digital counting operations. By using binary-weighted current summation and digital counters instead of analog-to-digital converters, the system dramatically reduces the number of passive components needed, thereby reducing chip area while maintaining computational accuracy.
Solution Approach 2:
The patent changes the operational parameters from high-precision analog conversions to discrete bitwise operations. By operating in the digital domain with binary-weighted currents and integer counting, the system eliminates the need for large ADC/DAC circuits, significantly reducing chip size while preserving the computational results.
3Measurement precision
If multi-bit inputs and weights are used for high accuracy MAC operations, then computational accuracy is improved, but IO bottleneck worsens and efficiency decreases
Solution Approach 1:
The patent segments multi-bit weight values into individual bits and processes them through separate bitwise multiplication operations. This segmentation enables parallel processing of multiple bit combinations, maintaining high computational accuracy while improving throughput by utilizing the memory array's parallelism across multiple operation cycles.
Solution Approach 2:
The patent maintains continuous useful action by immediately reusing the memory array and current pathways for subsequent bitwise operations after each multiplication. Instead of waiting for complete conversion or sequential processing, the system continuously performs multiple bitwise MAC operations in rapid succession, eliminating idle time and improving overall efficiency.
Data Source
AI summary
A memory device and an operation method thereof are provided. The memory device includes: a memory array including a plurality of memory cells for storing a plurality of weights; a multiplication circuit coupled to the memory array, for performing bitwise multiplication on a plurality of input data and the weights to generate a plurality of multiplication results; a counting unit coupled to the multiplication circuit, for performing bitwise counting on the multiplication results to generate a MAC (multiplication and accumulation) operation result.