Matrix Compression Accelerator Using 2D-to-1D Memory Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing technologies face inefficiencies in transferring large matrices between memories of different access times, particularly between slower external data memory (DRAM) and faster local data memory (SRAM), which hampers the processing speed of convolutional neural networks (CNNs) due to memory alignment and transfer length restrictions.
Innovation Solution
A matrix compression accelerator (MCA) system that implements lossless data compression (LDC) and decompression (LDD) using a multi-stage process, transforming 2D data into 1D and back, optimized for efficient data movement between external and local data memories, facilitated by a direct memory access (DMA) controller to minimize memory resources and overlap compute/transfer timing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transferred directly between external data memory (EDM) and local data memory (LDM) without compression, then the transfer process is simple and fast per access, but the total transfer time increases due to the large volume of data that must be moved across the slow EDM interface
Solution Approach 1:
The patent segments the data transfer process into multiple stages: compression stage (compressing data in LDM before transfer), transfer stage (moving compressed data from LDM to EDM), and decompression stage (recompressing data in EDM when needed). This segmentation allows the system to optimize each stage independently, using fast LDM for compression/decompression operations and only utilizing the slow EDM interface for actual data transfer, thereby resolving the contradiction between transfer simplicity and total transfer time.
Solution Approach 2:
The patent applies preliminary action by performing data compression in LDM before the data is transferred to EDM. By pre-compressing the data in the fast local memory, the system reduces the volume of data that needs to be transferred across the slow external memory interface, thereby significantly reducing total transfer time while maintaining operational simplicity.
2Loss of substance
If data compression and decompression operations are performed, then the total data volume to be transferred is reduced, but the complexity of the transfer system increases due to additional processing stages and control logic
Solution Approach 1:
The patent merges the compression/decompression functionality directly into the data transfer processor (DTP) that already exists in the system. By integrating these functions within the existing DTP architecture rather than adding separate dedicated compression/decompression units, the system reduces data volume effectively while minimizing the increase in overall system complexity.
Solution Approach 2:
The patent implements self-service by enabling the DTP to automatically perform compression and decompression operations as part of its normal data transfer workflow. The DTP autonomously determines when compression is beneficial and executes the appropriate operations without requiring external control or additional complex coordination logic, thereby reducing data volume while keeping the control system relatively simple.
3Ease of operation
If the DTP sequentially executes compression, transfer, and decompression operations, then the processing is straightforward and easy to control, but the total processing time increases due to lack of parallelism
Solution Approach 1:
The patent applies continuity of useful action by enabling overlapping of DTP operations with MCE compute operations. While the DTP is performing compression, transfer, or decompression operations, the MCE can simultaneously perform matrix computations on already-prepared data. This overlapping ensures that both subsystems are continuously productive rather than idle, reducing total processing time while maintaining straightforward sequential control within each subsystem.
Data Source
AI summary
A matrix compression/decompression accelerator (MCA) system/method that coordinates lossless data compression (LDC) and lossless data decompression (LDD) transfers between an external data memory (EDM) and a local data memory (LDM) is disclosed. The system implements LDC using a 2D-to-1D transformation of 2D uncompressed data blocks (2DU) within LDM to generate 1D uncompressed data blocks (1DU). The 1DU is then compressed to generate a 1D compressed superblock (CSB) in LDM. This LDM CSB may then be written to EDM with a reduced number of EDM bus cycles. The system implements LDD using decompression of CSB data retrieved from EDM to generate a 1D decompressed data block (1DD) in LDM. A 1D-to-2D transformation is then applied to the LDM 1DD to generate a 2D decompressed data block (2DD) in LDM. This 2DD may then be operated on by a matrix compute engine (MCE) using a variety of function operators.