BEICSR Format for Neural Network Accelerator Sparsity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in fully exploiting the sparsity of intermediate features in deep Graph Convolutional Networks (GCNs) for accelerating neural networks, leading to inefficiencies in memory access and computational workload.
Innovation Solution
The proposed solution involves an apparatus for accelerating neural networks that utilizes a bitmap-index embedded in-place Compressed Sparse Row (BEICSR) format for compressing intermediate feature sparsity, minimizing off-chip memory accesses, and handling varying working set sizes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing sparse formats such as CSR are used for intermediate features, then the format compatibility is maintained, but the compression rate and memory access efficiency deteriorate
Solution Approach 1:
The patent changes the parameter representation from traditional CSR format to a bitmap-indexed compressed sparse row format. This transformation modifies how sparse data is encoded, using bitmaps to mark non-zero element positions, thereby reducing the memory footprint and improving compression rate while maintaining compatibility with sparse matrix operations.
Solution Approach 2:
The patent extracts and separates the index information from the data values in sparse matrices. By using bitmap indices to represent the positions of non-zero elements independently from their values, the system achieves more efficient memory access patterns and reduces the overall memory traffic required to process intermediate features.
2Power
If DNN accelerators are designed to reduce computational workload, then the computational complexity is reduced, but the memory traffic volume remains high
Solution Approach 1:
The patent applies preliminary compression to intermediate features using the bitmap-indexed CSR format before they are processed by the accelerator. This pre-processing step reduces the volume of data that needs to be transferred between memory and computing units, thereby reducing memory traffic volume while maintaining the computational efficiency benefits of the accelerator.
3Quantity of substance
If tiling techniques are used to reduce working set size, then the cache utilization is improved, but the handling of varying sparsity levels becomes difficult
Solution Approach 1:
The patent implements a dynamic sparsity-aware tiling strategy where the tile size and compression level are adjusted based on the actual sparsity characteristics of the intermediate features. This dynamic adaptation allows the system to optimize cache utilization for each specific computation while maintaining effectiveness across varying sparsity levels throughout the network.
Data Source
AI summary
An apparatus for accelerating neural networks, includes: a memory for storing graph input data including vertices and edges; an aggregation engine that processes the accumulation of features and generates feature vectors by taking the graph input data and performing an aggregation operation on the graph input data; an on-chip cache for caching the feature vectors; and a combination engine that generates a systolic array for matrix multiplications based on the feature vectors taken from the on-chip cache and weights taken from the memory.