Crossbar Transpose Memory for Low-Latency Matrix Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix transposition operations in neural networks incur significant memory access latencies and overhead due to the need for individual memory transactions, especially when transposing large data arrays, which can result in substantial latency and inefficiency.
Innovation Solution
Utilize a crossbar with an internal transpose memory to rearrange data arrays on-the-fly, allowing parallel data transfer and write operations directly to a target memory, reducing the need for individual DMA transactions and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If individual memory transactions are used for each data element during matrix transposition, then data can be transferred to target memory, but memory access latency and overhead increase significantly
Solution Approach 1:
The patent segments the large data array into multiple rows that can be processed in parallel. Each row is transferred independently through separate memory transactions, allowing the transposition operation to be divided into manageable chunks that can be handled concurrently by multiple crossbar output ports, thereby reducing overall latency while maintaining data integrity.
Solution Approach 2:
The patent transforms the transposition operation by changing the dimensional approach: instead of transferring data element-by-element in sequence, it transfers entire rows in parallel across multiple dimensions (multiple output ports simultaneously). This dimensional change allows N rows to be transposed in N parallel transactions rather than requiring N² sequential element transfers.
2Productivity
If traditional DMA transactions are used for matrix transposition, then data transfer can be performed, but substantial overhead is incurred due to multiple individual transactions
Solution Approach 1:
The patent merges multiple individual data element transfers into consolidated row-level transactions. By combining N data elements into a single row transaction and using parallel crossbar paths, it reduces the total number of DMA transactions from N² to N, thereby improving throughput while managing complexity through the crossbar's parallel architecture.
Solution Approach 2:
The crossbar structure acts as an intermediary between the source and target memories, enabling parallel row transfers without requiring multiple sequential DMA transactions. The crossbar absorbs the complexity of routing N rows to N different target locations simultaneously, simplifying the overall transfer mechanism while maintaining high productivity.
3Loss of time
If data is transposed using sequential memory access, then memory operations can be completed, but execution time increases due to lack of parallelism
Solution Approach 1:
The patent applies preliminary action by pre-organizing data into row structures before transfer. Each row is prepared and staged for parallel transmission through the crossbar, allowing the transposition operation to begin immediately with multiple simultaneous transfers rather than sequential processing, thereby reducing execution time while maintaining operational simplicity.
Solution Approach 2:
The patent introduces dynamics by enabling the crossbar to dynamically route multiple rows to different target memory locations in parallel. The system transitions from static sequential access to dynamic parallel routing, where N rows can be simultaneously directed to N different output ports, reducing execution time while preserving ease of operation through automated routing logic.
Data Source
AI summary
Techniques to perform transpose operations in a crossbar circuit may include receiving, for a transpose write operation to transpose a data array, a set of write transactions from one or more data sources. Each write transaction can include an opcode and a row size of the data array being transposed. Data is written into the transpose memory in response to write transactions having an opcode indicating that the write transaction contains row data of the data array being transposed. When it is determined that the data array has been written into the transpose memory, the data array is outputted from the transposed memory in a transposed format to write the data array to a target memory.