Matrix Compression Accelerator for Faster EDM-LDM Transfers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing technologies face inefficiencies in transferring large matrices between memories of different access times, particularly between slower external data memory (DRAM) and faster local data memory (SRAM), which hampers the processing speed of convolutional neural networks (CNNs) due to memory alignment and transfer length restrictions.
Innovation Solution
A matrix compression accelerator (MCA) system that implements lossless data compression and decompression using a multi-stage process, including 2D-to-1D transformation and 1D-to-2D transformation, facilitated by a direct memory access (DMA) controller, to optimize data transfers between external and local data memories, thereby reducing the number of memory accesses and enhancing processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred directly between external data memory (EDM) and local data memory (LDM) without compression, then the transfer process is simple and fast per operation, but the total transfer time increases due to the large volume of data requiring multiple slow EDM accesses
Solution Approach 1:
The patent segments the data transfer process into multiple stages: compression stage (dividing data into blocks, applying compression algorithms), transfer stage (moving compressed data), and decompression stage (restoring original data). This segmentation allows optimization of each stage independently, particularly reducing the volume of data transferred between EDM and LDM through compression.
Solution Approach 2:
The patent applies preliminary compression action to data before transfer between EDM and LDM. By compressing data blocks before they leave EDM, the system reduces the total volume of data that must be transferred, thereby reducing the number of slow EDM access operations required and decreasing total transfer time.
2Quantity of substance
If compression and decompression operations are performed, then the volume of data transferred is reduced and fewer memory accesses are needed, but the complexity of the transfer system increases
Solution Approach 1:
The patent introduces compression and decompression modules as intermediary components between EDM and LDM. These intermediaries process data blocks through standardized algorithms (e.g., run-length encoding, Huffman coding) that reduce data volume while maintaining manageable system complexity through modular design and reuse of compression/decompression functionality across multiple transfer operations.
3Speed
If the data transfer interface implements multi-stage compression and transformation processes, then the compute/transfer timing overlap is improved and processing speed increases, but the operation timing within LDM increases
Solution Approach 1:
The patent implements continuous operation of the data transfer interface by maintaining multi-stage compression and transformation processes active during data transfers. The system continuously compresses data blocks, transfers them, and decompresses them in an overlapping manner with compute operations, ensuring the LDM remains continuously utilized without idle等待 periods, thereby improving overall processing speed despite increased per-operation timing.
Data Source
AI summary
A matrix compression/decompression accelerator (MCA) system/method that coordinates lossless data compression (LDC) and lossless data decompression (LDD) transfers between an external data memory (EDM) and a local data memory (LDM) is disclosed. The system implements LDC using a 2D-to-1D transformation of 2D uncompressed data blocks (2DU) within LDM to generate 1D uncompressed data blocks (1DU). The 1DU is then compressed to generate a 1D compressed superblock (CSB) in LDM. This LDM CSB may then be written to EDM with a reduced number of EDM bus cycles. The system implements LDD using decompression of CSB data retrieved from EDM to generate a 1D decompressed data block (1DD) in LDM. A 1D-to-2D transformation is then applied to the LDM 1DD to generate a 2D decompressed data block (2DD) in LDM. This 2DD may then be operated on by a matrix compute engine (MCE) using a variety of function operators.