Matrix DMA Controller Compresses Sparse Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data processing systems face inefficiencies in handling large matrices due to the high bandwidth requirements for transferring sparse and near-sparse matrices, which can slow down operations and reduce performance, especially when dealing with finite precision mathematical operations that result in small non-zero values instead of zeros.
Innovation Solution
Incorporating a direct memory access controller that performs compression and decompression of matrices during transfer between external and internal memory, utilizing a matrix DMA controller to read and write compressed matrices, and employing a compress module that sets small values to zero, thereby reducing data transfer time and bandwidth usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If sparse matrices are transferred between external and internal memory without compression, then data transfer accuracy is maintained, but data transfer time and bandwidth usage increase significantly
Solution Approach 1:
The matrix data is segmented into non-zero elements and zero elements. Only non-zero elements are stored and transferred in compressed format, while zero elements are represented implicitly. This segmentation reduces the amount of data that needs to be transferred between external and internal memory, directly addressing the bandwidth and time issues.
Solution Approach 2:
A compression module and decompression module are introduced as intermediary components between external memory and the processing units. The compression module compresses matrices before storage in external memory, and the decompression module decompresses them during transfer to internal memory, reducing the actual data transfer volume while maintaining processing accuracy.
2Productivity
If finite precision mathematical operations are performed on matrices, then computation speed is improved, but small non-zero values are produced instead of exact zeros
Solution Approach 1:
A thresholding operation is performed preliminarily on the result matrix to identify and set small non-zero values (below a certain threshold) to exact zeros. This preliminary action corrects the precision issue caused by finite precision arithmetic while maintaining the overall computation speed, as the thresholding operation is more efficient than full-precision computation.
Solution Approach 2:
The precision parameter of the computation is adjusted by introducing a threshold value. Instead of requiring exact zero representation, the system accepts values below the threshold as effectively zero. This parameter change allows the use of finite precision arithmetic while maintaining acceptable accuracy for sparse matrix operations.
3Loss of energy
If compressed matrix storage is implemented, then memory bandwidth usage is reduced, but additional compression and decompression processing is required
Solution Approach 1:
The compression and decompression operations are integrated into the memory controller and DMA system, allowing the memory subsystem to serve its own compression needs without burdening the main processor. This self-service approach reduces the processing complexity burden on the CPU while still achieving bandwidth savings.
Solution Approach 2:
The compression and decompression functions are merged with the memory control operations. The compression module works in conjunction with the memory controller to compress data before it leaves external memory, and the decompression module is integrated with the memory interface to decompress data as it enters internal memory. This merging reduces the overall system complexity compared to having separate compression and memory management systems.
Data Source
AI summary
An electronic device includes a first memory subsystem, a second memory subsystem and a direct memory access controller. In response to a first type of request from a processor, the direct memory access controller requests data from the first memory subsystem and provides the data to the second memory subsystem. In response to a second type of request from a processor, the direct memory access controller requests an uncompressed matrix from the first memory subsystem, compresses the uncompressed matrix to generate a compressed matrix, and provides the compressed matrix to the second memory subsystem. In response to a third type of request from a processor, the direct memory access controller requests a compressed matrix from the second memory subsystem, un-compresses the compressed matric to generate an uncompressed matrix, and provides the un-compressed matrix to the first memory subsystem.