Matrix Multiplication Tile Mapping on Parallel Processors

The patent segments the matrix operation into multiple independent thread groups, where each thread group processes a portion of the matrix multiplication. This allows parallel execution of multiple thread groups on the same microprocessor, effectively increasing throughput without requiring a more complex processor architecture. The segmentation of work enables the system to overcome the limitation of limited arithmetic and memory access logic by distributing computations across multiple concurrent execution units.

The patent implements continuous useful action by overlapping memory access operations with computation operations within each thread group. While some threads are computing matrix elements, other threads are simultaneously loading data from memory. This continuous utilization of processing resources eliminates idle time and maximizes the effective performance of the single microprocessor, achieving multiprocessing-like efficiency without the associated cost.

The patent applies preliminary action by pre-loading matrix data into fast on-chip memory (such as cache or register memory) before the computation phase begins. Thread groups are organized to efficiently utilize this pre-loaded data, minimizing the need for repeated accesses to slower global memory. This preliminary data preparation reduces the memory bandwidth burden during the actual matrix multiplication, allowing the GPU to achieve high computational bandwidth without being bottlenecked by memory bandwidth limitations.