Prefetch Buffer Matrix Multiplication Circuit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix multiplication in computing systems is inefficient due to high processing time and memory bottlenecks caused by the need for multiple memory fetches during parallel multiplication operations, leading to increased power consumption.
Innovation Solution
A matrix multiplication circuit that includes prefetch buffers and partial products buffers to reduce memory fetches to one per cycle by prefetching and parallel processing of matrix elements, allowing for efficient computation of partial products and subsequent storage in result buffers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If hardware accelerators are used to perform parallel multiplication operations, then processing speed is improved, but the number of memory fetches increases causing IO bottlenecks
Solution Approach 1:
The patent applies preliminary action by prefetching matrix elements from memory into buffer memory before they are needed for multiplication operations. This allows the parallel multiply circuit to perform computations without waiting for memory fetches, thereby maintaining high processing speed while reducing the frequency of memory access operations and eliminating IO bottlenecks.
2Quantity of substance
If multiple memory fetches are performed for parallel multiplication, then more data is available for computation, but power consumption increases
Solution Approach 1:
The patent prefetches required matrix elements into buffer memory in advance, ensuring that sufficient data is available for parallel multiplication operations. By having data pre-loaded in buffer memory, the system avoids multiple repeated memory fetches, thereby reducing power consumption while maintaining data availability for computation.
Solution Approach 2:
The patent introduces buffer memory as an intermediary between main memory and the parallel multiply circuit. This buffer memory stores prefetched matrix elements locally, allowing the computation unit to access data without directly querying main memory repeatedly. The buffer acts as a mediator that satisfies data requirements while minimizing expensive and power-intensive memory access operations.
3Productivity
If parallel multiplication operations are implemented, then computational throughput is improved, but memory access complexity increases
Solution Approach 1:
The patent segments the memory access function into two distinct parts: prefetching operations that load data into buffer memory, and computation operations that use the buffered data. This segmentation allows the complex memory access requirements of parallel multiplication to be broken down into simpler, manageable stages, reducing overall memory access complexity while maintaining high computational throughput.
Data Source
AI summary
A matrix multiplication circuit comprises a memory storage device, processing circuitry, a parallel multiply circuit, and buffer circuits. The parallel multiply circuit simultaneously performs a count of multiplies in a parallel multiplication operation. The buffer circuits include prefetch buffer circuits each having a storage array dimension corresponding to the count of multiplies in the parallel multiplication operation. The processing circuitry loads a first prefetch buffer circuit with values from the first matrix; fetches a value of the second matrix and, in parallel with the fetch, preload the second prefetch buffer circuit with another value from the first matrix; initiates a parallel multiply of the fetched value of the second matrix and the values in the first prefetch buffer circuit; and stores partial product results of the parallel multiply, including adding a current partial product result to a previously stored partial product result.