Crossbar Transpose Memory for Parallel Matrix Data Transfers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix transposition operations in neural networks incur significant memory access latencies and overhead due to the need for individual memory transactions for each data element, especially when transposing large data arrays, leading to inefficient data transfer and increased execution time.
Innovation Solution
Utilize a crossbar with an internal transpose memory to rearrange data arrays on-the-fly during write operations, storing data diagonally in the transpose memory and rotating rows to write transposed data in parallel to the target memory, reducing the need for individual DMA transactions and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If individual memory transactions are used for each data element during matrix transposition, then data can be transferred to target memory, but memory access latency and overhead increase significantly
Solution Approach 1:
The patent segments the large data array into multiple rows that can be processed in parallel. Each row is transferred independently through separate memory transactions, allowing concurrent processing of multiple rows while reducing the total number of transactions compared to element-by-element transfer. This segmentation enables the crossbar to distribute row transfers across multiple output ports simultaneously.
Solution Approach 2:
The patent transforms the transposition operation by changing the dimension of parallelism from element-level to row-level. Instead of transferring one element at a time through a single channel, the system transfers entire rows in parallel through multiple channels of the crossbar, effectively adding a spatial dimension to the data transfer operation and achieving full bandwidth utilization.
2Productivity
If data arrays are transposed using traditional DMA transactions, then data transfer can be performed, but the number of transactions increases significantly for large arrays
Solution Approach 1:
The patent merges multiple individual data element transfers into single row-level transfers. By combining adjacent data elements into complete rows that are transferred together in parallel through the crossbar, the system reduces the total number of DMA transactions required while maintaining high data transfer productivity.
Solution Approach 2:
The patent performs preliminary organization of data into row structures before transfer. The source memory is arranged such that complete rows are ready for parallel transfer, and the crossbar is pre-configured with the transposition mapping. This preliminary preparation enables efficient parallel execution without requiring intermediate transactions for each element.
3Reliability
If data is stored and transferred element-by-element, then transposition can be achieved, but execution time increases due to sequential processing
Solution Approach 1:
The patent introduces dynamic parallelism by configuring the crossbar to simultaneously route multiple rows to different output ports based on their destination addresses. The system dynamically adjusts the transfer pattern to maintain correct transposition relationships while executing multiple operations in parallel, thereby reducing execution time without sacrificing operational correctness.
Data Source
AI summary
Techniques to perform transpose operations in a crossbar circuit may include receiving a set of write transactions to write a data array to a target memory, and determining that the set of write transactions is for a transpose write. Write data for each of the write transactions can be stored diagonally in a transpose memory of the crossbar circuit. Each row of data in the transpose memory can be rotated, and rotated data from each row of the transpose memory can be provided to a corresponding output port of the crossbar circuit to write to the target memory.