Compute-in-Memory Block Reuse for Low-Energy Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional matrix-matrix multiplication methods in computing architectures result in significant power consumption due to inefficient memory access, particularly in large-sized matrices used in scientific and engineering applications like neural networks.
Innovation Solution
A computing system with a Compute-in-Memory (CIM) design and a two-level memory system is employed, utilizing a processor control unit to manage A and B block buffering in L1 and CIM memories, selecting an optimal reuse scheme based on matrix sizes, sparsity, and buffering capabilities to minimize memory access energy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If conventional memory access methods are used for large-sized matrix multiplication, then computation can be performed, but power consumption is significant due to inefficient memory access
Solution Approach 1:
The patent divides large matrices into blocks and organizes them in a two-level memory system. The L1 memory stores frequently accessed A blocks while the CIM memory stores B blocks, segmenting the memory access patterns to reduce redundant transfers and improve energy efficiency.
Solution Approach 2:
The patent transitions from conventional von Neumann architecture to Compute-in-Memory (CIM) architecture, adding a computational dimension to the memory system. This allows multiplication operations to occur directly within the memory device, eliminating the need for data to move between memory and processing units.
2Adaptability or versatility
If A blocks are buffered in L1 memory and programmed to CIM memory for each B block processing, then B block reuse is enabled, but L1 memory access energy is consumed
Solution Approach 1:
The patent pre-loads A blocks into the L1 memory buffer before processing multiple B blocks. This preliminary action allows the same A blocks to be reused across multiple B block processing operations without repeated L1 memory accesses, reducing energy consumption.
Solution Approach 2:
The patent maintains A blocks in the L1 memory buffer during processing, effectively recovering them for reuse rather than discarding them after each B block operation. This recovery strategy enables multiple B blocks to be processed using the same A blocks without additional L1 memory accesses.
3Reliability
If B blocks are continuously loaded from L2 memory to CIM memory, then fresh data is available for computation, but L2 memory access energy increases
Solution Approach 1:
The patent enables continuous processing by maintaining A blocks in the L1 buffer, allowing multiple B blocks to be processed sequentially without requiring repeated L1 memory accesses. This continuity reduces the frequency of high-energy L2 memory accesses while ensuring data availability.
Data Source
AI summary
A computing system with power-saving compute-in-memory (CIM) design that minimizes the computation energy of the matrix-matrix multiplication is shown. A processor control unit loads A blocks divided from a matrix AM×K from a second-level (L2) memory to a first-level (L1) memory, and loads B blocks divided from a matrix BK×N from the L2 memory to a CIM memory. The A blocks buffered in the L1 memory are programmed to a register file to be entered into the CIM memory. The CIM memory performs multiply-and-accumulate (MAC) calculations on the A blocks and the B blocks to generate C blocks which form a matrix CM×N (=AM×K×BK×N). Based on the size of AM×K and BK×N, an A block buffering capability of the L1 memory, and a B block buffering capability of the CIM memory, the reuse scheme is properly selected to reuse the buffered A blocks and B blocks.