Asymmetric Scratchpad Memory Allocation for Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiplication methods, particularly in architectures with scratchpad memories, face inefficiencies due to symmetric memory allocation and high memory access costs, which hinder performance improvements despite advancements in computational power.
Innovation Solution
The proposed solution involves asymmetric allocation of scratchpad memory space for input matrices and the product matrix, optimizing tile sizes and access patterns to minimize memory traffic, and utilizing a scratchpad-based architecture with DMA support to enhance matrix multiplication efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If symmetric memory allocation is used for matrix multiplication, then memory management is simplified, but memory bandwidth efficiency is reduced and performance deteriorates
Solution Approach 1:
The patent applies asymmetry by allocating different portions of scratchpad memory to different matrices based on their size requirements. Specifically, it allocates a first portion of scratchpad memory for a first matrix and a second portion for a second matrix, where the allocation ratios differ from equal division. This asymmetric allocation optimizes memory bandwidth utilization and improves matrix multiplication performance while maintaining manageable memory control through systematic allocation rules.
Solution Approach 2:
The patent implements local quality by tailoring the memory allocation strategy to the specific characteristics of each matrix being multiplied. Rather than applying a uniform allocation policy, the system adjusts the scratchpad memory portion assigned to each matrix based on its dimensions and computational requirements, thereby optimizing local memory efficiency for each matrix operation.
2Power
If computational power is increased exponentially, then processing capability is improved, but memory access costs become the bottleneck and performance is hindered
Solution Approach 1:
The patent applies preliminary action by pre-loading matrix data into scratchpad memory before the actual multiplication computation begins. By allocating and preparing the scratchpad memory portions for storing matrix data in advance, the system reduces memory access time during the computational phase, thereby mitigating the memory bottleneck that would otherwise limit the effectiveness of high computational power.
Solution Approach 2:
The patent introduces scratchpad memory as an intermediary between main memory and the processing units. This intermediate storage layer allows frequently accessed matrix data to be held closer to the computational units, reducing the time penalty associated with accessing distant memory resources and enabling the computational power to be fully utilized without being bottlenecked by memory access speeds.
3Productivity
If tile-based approach is used, then data reuse is increased, but memory allocation complexity increases and asymmetric matrices are inefficiently utilized
Solution Approach 1:
The patent applies dynamics by making the scratchpad memory allocation flexible and adaptive to the specific dimensions and characteristics of the matrices being multiplied. Rather than using fixed tile sizes and static allocation schemes, the system dynamically adjusts the portion of scratchpad memory assigned to each matrix based on runtime parameters, thereby maintaining data reuse efficiency while reducing allocation complexity and improving handling of asymmetric matrices.
Data Source
AI summary
Techniques are described herein for performing efficient matrix multiplication in architectures with scratchpad memories or associative caches using asymmetric allocation of space for the different matrices. The system receives a left matrix and a right matrix. In an embodiment, the system allocates, in a scratchpad memory, asymmetric memory space for tiles for each of the two matrices as well as a dot product matrix. The system proceeds with then performing dot product matrix multiplication involving the tiles of the left and the right matrices, storing resulting dot product values in corresponding allocated dot product matrix tiles. The system then proceeds to write the stored dot product values from the scratchpad memory into main memory.