Matrix Computing Device With Scratchpad Memory For Flexible Widths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing devices face inefficiencies in performing matrix operations due to constraints imposed by inter-chip communication, insufficient on-chip cache, and inflexible supported matrix lengths, leading to suboptimal performance in algorithms that rely heavily on matrix computations.
Innovation Solution
A matrix computing device is designed with a storage unit, register unit, and matrix operation unit that temporarily stores matrix data in a scratchpad memory, allowing for flexible support of different matrix widths and sizes, and featuring a simplified instruction set architecture to enhance performance in executing matrix operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a general-purpose processor is used for matrix operations, then the device can perform scalar computations, but the operation performance is low due to single-processor limitation and intercommunication bottlenecks
Solution Approach 1:
The processor is divided into multiple computing cores (first computing core, second computing core, etc.), each capable of independently executing matrix operations. This segmentation enables parallel processing of matrix data, significantly improving computational performance while maintaining manageable device complexity through modular architecture
Solution Approach 2:
Multiple computing cores are combined with a shared memory unit that stores matrix data. This merging allows all cores to access the same matrix data simultaneously, enabling efficient parallel computation without the intercommunication bottlenecks of distributed systems, thus improving matrix operation performance
2Productivity
If multiple general-purpose processors are used for parallel operations, then the computing power increases, but intercommunication among processors becomes a performance bottleneck
Solution Approach 1:
Multiple computing cores are merged into a single processor architecture with a shared memory unit, eliminating the need for inter-processor communication. All cores access matrix data directly from the shared memory, reducing intercommunication time and improving overall computing efficiency for matrix operations
3Productivity
If a GPU with small on-chip cache is used, then matrix computation can be performed, but constant transportation of off-chip data makes off-chip bandwidth a main performance bottleneck
Solution Approach 1:
Matrix data is pre-loaded into the shared memory unit before computation begins. This preliminary action ensures that all computing cores have immediate access to the required data without needing to repeatedly fetch from off-chip memory during computation, reducing data transportation energy and improving computation efficiency
4Productivity
If a specialized matrix operation device with customized register file is used, then matrix operations can be performed, but the device is limited by the register file and unable to flexibly support matrix operations of different lengths
Solution Approach 1:
The shared memory unit serves multiple functions: it acts as both the data storage unit and the addressing unit for matrix operations. This universal design allows the device to handle matrix operations of various lengths and dimensions by simply changing the address pointers, providing flexibility without requiring specialized register files for different matrix sizes
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to a matrix computing device comprising a storage unit, a register unit and a matrix operation unit, wherein matrices are stored in the storage unit, matrix storage addresses are stored in the register unit, and the matrix operation unit acquires a matrix address from the register unit according to a matrix operation instruction, and then acquires a corresponding matrix from the storage unit according to the matrix address, subsequently performs a matrix operation according to the matrix acquired so as to obtain a matrix operation result. The device temporarily stores matrix data involved in computation in a scratchpad memory, so that different data widths can be supported more flexibly and effectively during matrix operations, and execution of computing tasks comprising a large number of matrices can be improved.