In-Memory Matrix Multiplication Architecture With Parallel Compute Cells
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional digital computing systems face limitations in performing matrix multiplication due to memory access constraints, which hinder real-time processing and increase power consumption, especially in applications requiring large matrices and high data rates.
Innovation Solution
A compact in-memory computer architecture is introduced, featuring memory components arranged in rows and columns with compute engines connected to bit cells, allowing for parallel processing and reduced physical hardware requirements, utilizing memory-select switches to isolate memory components and enabling efficient matrix multiplication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in digital memories (SRAM/DRAM) and accessed through memory-and-address bus, then data storage capacity is improved, but access speed and processing efficiency deteriorate due to bus bit width limitations
Solution Approach 1:
The patent segments the monolithic memory architecture into multiple memory banks that can be accessed independently and in parallel. Each memory bank has its own compute engine, allowing simultaneous data retrieval and processing operations across multiple banks, thereby increasing effective data access speed without reducing storage capacity.
Solution Approach 2:
The patent merges memory storage functions with compute processing functions into integrated memory-compute units. Each memory bank is directly coupled with a compute engine, eliminating the need for separate memory access and processing stages, thus reducing latency and improving overall data throughput.
2Measurement precision
If conventional digital computing systems perform matrix multiplication with large matrices, then computation accuracy is improved, but power consumption increases due to repeated memory access operations
Solution Approach 1:
The patent performs preliminary data loading by pre-fetching required matrix data into the memory banks before computation begins. The compute engines then process the data locally without requiring repeated accesses to external memory, significantly reducing power consumption while maintaining full computation accuracy through precise arithmetic operations.
Solution Approach 2:
The patent introduces local compute engines as intermediaries between main memory and processing units. These compute engines cache intermediate computation results in local registers and buffers, reducing the frequency of high-power memory access operations while preserving computational accuracy through buffered arithmetic operations.
3Speed
If more hardware resources are allocated for high-speed matrix multiplication, then processing speed is improved, but device area and cost increase
Solution Approach 1:
The patent designs universal memory banks that can store different types of data (input matrices, weight matrices, intermediate results) and universal compute engines that can perform various operations (matrix multiplication, convolution, activation functions). This multi-functionality allows high processing speeds to be achieved without proportionally increasing hardware area, as the same physical resources serve multiple computational purposes.
Solution Approach 2:
The patent transitions from a two-dimensional memory layout to a three-dimensional stacked architecture where multiple memory banks and compute engines are vertically integrated. This dimensional change increases processing throughput by enabling parallel operations across multiple layers while maintaining a compact footprint on the semiconductor substrate.
Data Source
AI summary
A compact in-memory computer architecture includes memory components arranged in rows and columns, bit lines each connecting a row of memory components, and word lines each connecting a column of memory components. Each memory component has a bit cell and a compute engine connected to the bit cell. The bit cell is operable to store a bit and the compute engine is operable to process the bit. Each bit line connects a respective row of memory components and is operable to provide a bit to each memory component in the row of memory components. Each word line connects a respective column of memory components and is operable to enable each memory component in the column of memory components to write a bit into each memory component in the column of memory components.