Transformer Acceleration Through Segmented Memory Banks

The memory system is segmented into multiple independent memory banks (first memory bank array and second memory bank array). Each memory bank can independently store and retrieve data, allowing parallel access operations. This segmentation eliminates data access collisions that would occur in a single memory bank system, thereby improving transformer operation speed and productivity without significantly increasing overall system complexity.

The patent extracts the data storage and retrieval functions into separate, dedicated memory bank arrays. The first memory bank array stores weight matrices while the second memory bank array stores key-value vector pairs. This extraction allows each memory bank to be optimized for its specific function, enabling faster data access and eliminating bottlenecks in the transformer operation pipeline.

The patent combines multiple memory bank arrays into a unified memory system architecture. The first and second memory bank arrays work together as an integrated system, with each array handling specific data types (weight matrices and key-value vector pairs respectively). This merging approach achieves high data access efficiency while managing complexity through systematic organization and clear functional separation.