Matrix Compression Accelerator for DRAM-SRAM Transfer Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing technologies face inefficiencies in transferring large matrices between data memories with different access times, particularly between slower external dynamic random access memory (DRAM) and faster local static random access memory (SRAM), which hampers the processing speed of convolutional neural networks (CNNs) due to memory alignment and transfer length constraints.
Innovation Solution
A matrix compression accelerator (MCA) system that implements lossless data compression and decompression using a multi-stage process, including 2D-to-1D transformation and 1D-to-2D transformation, facilitated by a direct memory access (DMA) controller, to optimize data transfers between external and local data memories, thereby reducing the number of memory accesses and enhancing processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred directly between external DRAM and local SRAM without compression, then the transfer process is simple, but the number of memory accesses increases and processing speed decreases
Solution Approach 1:
The patent segments the data transfer process into distinct phases: compression phase (transferring compressed data from SRAM to DRAM) and decompression phase (transferring decompressed data from DRAM to SRAM). This segmentation allows compute operations to overlap with transfer operations, improving productivity while managing complexity through structured process division.
Solution Approach 2:
The patent applies preliminary compression to data before transferring it to external memory. By compressing data in advance during the transfer to DRAM, the system reduces the volume of data that needs to be transferred back, thereby reducing the number of memory accesses and improving processing speed without requiring the full uncompressed dataset to be resident in fast memory.
2Loss of time
If lossless data compression is implemented during data transfer, then the number of memory accesses is reduced, but the compression and decompression process time increases
Solution Approach 1:
The patent ensures continuity of useful action by overlapping compute operations with compression and decompression operations. While data is being compressed during transfer to DRAM or decompressed during transfer from DRAM, the matrix compute engine continues to perform computations on available data. This eliminates idle time and ensures that the compression/decompression overhead does not reduce overall productivity.
Solution Approach 2:
The patent introduces a data transfer processor as an intermediary component that handles compression and decompression operations. This dedicated processor acts as a mediator between the compute engine and external memory, managing the compression/decompression complexity separately from the main compute operations. The data transfer processor coordinates with the compute engine to enable overlapping operations and manages the compression state information.
3Quantity of substance
If data is compressed before transfer to external memory, then storage space is optimized, but data alignment and transfer length constraints must be satisfied
Solution Approach 1:
The patent employs dynamic adjustment of compression parameters and block sizes to satisfy data alignment and transfer length constraints. The system can adaptively choose compression block sizes that are multiples of the external memory transfer width (e.g., 128 bytes) and adjust compression ratios based on data characteristics. This dynamic approach allows the system to optimize data volume reduction while maintaining compatibility with memory interface constraints.
Solution Approach 2:
The patent changes parameters such as compression block size, transfer width, and alignment boundaries to optimize the balance between compression efficiency and memory transfer constraints. By adjusting these parameters, the system can achieve better compression ratios when data characteristics permit, or use larger block sizes when alignment constraints are more restrictive, thereby managing the trade-off between data volume reduction and transfer efficiency.
Data Source
AI summary
A matrix compression/decompression accelerator (MCA) system/method that coordinates lossless data compression (LDC) and lossless data decompression (LDD) transfers between an external data memory (EDM) and a local data memory (LDM) is disclosed. The system implements LDC using a 2D-to-1D transformation of 2D uncompressed data blocks (2DU) within LDM to generate 1D uncompressed data blocks (1DU). The 1DU is then compressed to generate a 1D compressed superblock (CSB) in LDM. This LDM CSB may then be written to EDM with a reduced number of EDM bus cycles. The system implements LDD using decompression of CSB data retrieved from EDM to generate a 1D decompressed data block (1DD) in LDM. A 1D-to-2D transformation is then applied to the LDM 1DD to generate a 2D decompressed data block (2DD) in LDM. This 2DD may then be operated on by a matrix compute engine (MCE) using a variety of function operators.