Neural Network Accelerator With Rank-Enhanced Block-Circulant Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing block-circulant matrix (BCM) compression methods for neural networks face limitations such as restricted representational capacity, inflexible compression ratios, and inefficient dataflow on resource-constrained FPGAs, leading to accuracy degradation and computational inefficiencies.
Innovation Solution
A rank-enhanced and highly-pruned block-circulant matrix (RP-BCM) framework that utilizes the Hadamard product to enhance matrix rank and implement BCM-wise pruning, along with a specialized dataflow method for BCM-compressed networks on FPGAs, incorporating a dedicated hardware accelerator.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If Block-Circulant Matrix (BCM) compression is applied to reduce memory complexity and computational overhead, then compression rate is improved, but accuracy is degraded due to limited representational capacity from rank constraints
Solution Approach 1:
The patent applies low-rank decomposition by factorizing the weight matrix into two lower-rank matrices (W = A × B), creating a composite structure that maintains representational capacity while reducing parameters. This composite approach allows the network to achieve higher compression rates without proportional accuracy loss, as the product of two low-rank matrices can approximate the original full-rank matrix more effectively than a single low-rank matrix.
Solution Approach 2:
The patent changes the parameter representation by introducing two sets of parameters (matrices A and B) instead of one, with each having fewer parameters than the original weight matrix. This parameter transformation enables the model to maintain expressive power through the interaction of multiple parameter sets, resolving the contradiction between compression and accuracy.
2Adaptability or versatility
If BCM size is increased to improve compression process flexibility, then compression ratio options increase, but accuracy drops significantly due to not considering weight importance
Solution Approach 1:
The patent applies local quality by differentiating the treatment of different weight components through low-rank decomposition. By factorizing the weight matrix into A and B, the method allows selective pruning and refinement of specific sub-matrices based on their importance, enabling flexible compression ratios while preserving critical weight information and maintaining accuracy.
Solution Approach 2:
The patent introduces dynamic flexibility by allowing the low-rank factorization to be applied at different stages of training and with varying ranks. The method enables adaptive adjustment of the decomposition rank and pruning ratios during training, providing dynamic control over the compression-accuracy tradeoff without sacrificing accuracy.
3Ease of manufacture
If standard BCM compression is applied to enable regular computational patterns, then hardware acceleration feasibility is improved, but accuracy is degraded due to structural constraints requiring most weights to be identical
Solution Approach 1:
The patent segments the weight matrix into two separate low-rank factors (A and B), each with reduced dimensions. This segmentation allows each factor to have its own structured form suitable for hardware acceleration while maintaining overall representational capacity, resolving the contradiction between hardware-friendly structure and accuracy.
Solution Approach 2:
The patent transitions from a single weight matrix W of size (m, n) to two factor matrices A (m, k) and B (k, n) where k < min(m, n). This dimensional transformation creates an intermediate dimension that enables both hardware-efficient computation and preserved accuracy by distributing the representational burden across multiple smaller matrices.
Data Source
AI summary
Disclosed is a computing system comprising a processor; and a neural network, wherein the processor obtains a third block circulant matrix, which is the Hadamard product of a first block circulant matrix and a second block circulant matrix from each layer of the neural network, trains the neural network by utilizing the third block circulant matrix as weights, and fine-tunes the first block circulant matrix and the second block circulant matrix by pruning a plurality of first sub-block circulant matrices included in the learned first block circulant matrix and a plurality of second sub-block circulant matrices included in the learned second block circulant matrix, respectively, for an arbitrary layer among layers of the neural network.