Matrix Transposition Apparatus for Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing artificial neural network processing apparatuses face performance degradation due to the increased hardware resources and execution time required for matrix transposition operations, especially when processing large matrices for Large Language Models.
Innovation Solution
A matrix transposition apparatus and method that utilize an on-chip memory module with a transposition operation function and a dedicated address generator, allowing for efficient matrix transposition without additional large hardware resources or execution time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a large buffer and many execution logics are used for fast transposition of large matrices, then transposition speed is improved, but hardware resources and execution time increase
Solution Approach 1:
The patent divides the large M×N matrix into multiple smaller sub-matrices of size (M/K)×N, where K is the number of memory banks. Each sub-matrix is processed independently through parallel transposition operations in different memory banks, enabling fast transposition without requiring a single large buffer. This segmentation allows the system to handle large matrices by breaking them into manageable chunks that can be processed in parallel.
Solution Approach 2:
The patent transforms the traditional 2D matrix transposition problem into a 3D processing scheme by introducing the memory bank dimension. Data is distributed across multiple memory banks (K banks), and transposition is achieved by coordinating reads and writes across this additional dimension. This allows parallel processing without requiring a large single-dimension buffer.
2Productivity
If a dedicated transposition buffer and execution logic are added, then transposition operation is accelerated, but processing time increases
Solution Approach 1:
The patent enables continuous data flow during transposition by having the data receiving unit continuously write transposed data to the matrix operator while the transposition operation progresses. The control unit continuously generates read and write address values, ensuring that the matrix transposition and subsequent matrix multiplication operations proceed without idle waiting time, thus improving overall productivity without increasing execution time.
3Reliability
If matrix data is stored and transposed using traditional methods, then correctness is maintained, but hardware resources and execution time increase for large matrices
Solution Approach 1:
The control unit implements feedback control by continuously monitoring and coordinating the read and write address generation across multiple memory banks. The control unit ensures that data is read from appropriate sources and written to correct destinations by managing the address sequences, maintaining transposition correctness while enabling parallel processing that improves productivity for large matrices.
Data Source
AI summary
An apparatus and a method for transposing matrix are disclosed. The apparatus includes a data interface that receives a plurality of matrix data included in an M×N matrix, a control unit that acquires a plurality of write address values and a plurality of read address values in accordance with a size of the plurality of matrix data and a size of the matrix data, a data receiving unit that stores the plurality of matrix data in accordance with the plurality of write address values in a memory module including a plurality of memory banks, and a transposed matrix generating unit that acquires a plurality of storage data from the memory module in accordance with the plurality of read address values, acquires the plurality of storage data as transposed matrix data, and stores the transposed matrix data in a matrix operator.